Code Soliloquies for Accurate Calculations in Large Language Models (Accepted at LAK'24)
This repository contains code for the paper: Code Soliloquies for Accurate Calculations in Large Language Models
Model: https://huggingface.co/luffycodes/higgs-llama-vicuna-ep25-70b
Demo can be found at the last of the readme.
To use the model, first install the fastchat library, and then follow the steps here:
- Update the conversation.py from our conversation_inference repository in the FastChat folder.
- Update the inference.py from our repository in the FastChat folder.
- Then run the model using the following command:
-
python3 -m fastchat.serve.cli --model-path luffycodes/higgs-llama-vicuna-ep25-70b --num-gpus 8 --temperature 0.7 --max-gpu-memory 20GiB - Note that code soliloquy is not implemented in UI
-
- Run the generate_conversations.py.
- Remember to put openai.api_key.
- Use the training instructions from fastchat library and run train_higgs_lora.py.
Descriptions of GPT-Tutorbot state prompts used to implement code soliloquy in conversation_gen
- deciding_state.txt = The 'deciding_state' is the first state of code soliloquy in which the GPT-tutorbot determines whether a calculation is needed for its response to the student. If it determines a calculation is needs is needed, it generates the description for Python code for performing that calculation and gpt-tutorbot transitions to 'use_python_state'; if it decides that no calculation is needed, the gpt-tutorbot transitions to 'no_python_state'.
- use_python_state.txt = In this state, gpt-tutorbot generates Python code based on Python code description generated during 'deciding_state'.
- received_python_state.txt = This is the final state of gpt-tutorbot soliloquy (when it is using python) in which it generate tutorbot's response based on Python code output.
- no_python_state.txt = This is the final state of gpt-tutorbot soliloquy (when it is not using python) in which it generate gpt-tutorbot's response.
If you use this work, please cite: Code Soliloquies for Accurate Calculations in Large Language Models https://arxiv.org/abs/2309.12161
@misc{sonkar2023code,
title={Code Soliloquies for Accurate Calculations in Large Language Models},
author={Shashank Sonkar and MyCo Le and Xinghe Chen and Lucy Liu and Debshila Basu Mallick and Richard G. Baraniuk},
year={2023},
eprint={2309.12161},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Demo:
