LLM Calculator#
The objective of this assignment is to deepen your understanding of large language models (LLMs) and explore how these models can be used to perform different functions, such as mathematical calculations. This exercise builds on the concepts covered in the previous LLM notebook and introduces practical challenges to enhance your skills.
Assignment#
Complete the following tasks, and submit your responses in a single notebook (.ipynb) containing code, explanations, and any visualisations (if applicable). The notebook should be well-commented and self-contained, so it can be executed on other machines without modifications.
1. Autoregressive Summation#
In the LLM notebook, we practised text generation using a simple dataset, such as Tiny Shakespeare. Here, the goal is to determine whether the same approach can be adapted to perform basic arithmetic—specifically, to compute the sum of two numbers, such as \(a + b = c\).
To complete this task:
Generate a dataset that includes a wide variety of summation examples, covering both positive and negative integers. You may choose to generate these examples dynamically in the code or create a static dataset saved in a file.
Create a validation set that remains constant across all experiments. This will enable consistent evaluation of model performance on different versions and settings.
Model adjustments: Consider whether any changes to the model architecture or hyperparameters are necessary to enable the model to learn summation.
Evaluation: Check if the model can learn the summation task effectively. Measure its accuracy on the validation set.
Reflect on the results and discuss the accuracy of the model in predicting the correct sums.
2. Subtraction#
Similar to the task above, investigate whether the model can learn subtraction, expressed as \(a - b = c\).
Dataset: Extend or modify your dataset to include subtraction examples with both positive and negative numbers.
Performance Evaluation: Assess the model’s accuracy on this new task.
Comparison: Does the model learn both summation and subtraction, or does it struggle to handle both operations simultaneously?
Provide insights into any challenges encountered and analyse whether the model is equally capable of learning both tasks.
Bonus Questions (Optional)#
These additional tasks are optional but offer extra points for those interested in exploring further.
3. Multiplication and Division#
Extend the model’s functionality to handle multiplication \(a \times b = c\) and division \(\frac{a}{b} = c\).
Dataset Expansion: Update your dataset to include examples of multiplication and division.
Challenges and Observations: Note any challenges in training the model on these operations and evaluate its performance.
4. More Advanced Calculator#
Investigate whether other mathematical operations or expressions (e.g., powers, roots, or mixed operations) can be incorporated into your model. Document your findings and explain any limitations you observe.