Natural language to command line

CLI Agent

A fine-tuned small language model that turns plain-English instructions into step-by-step command-line plans, with dry-run execution and safety validation.

0.125 → 0.318

BLEU

0.203 → 0.445

ROUGE-L

6 of 7

Agent test cases passed

The problem

General small models produce vague or wrong shell commands. The goal was to see how far a 1.1B-parameter model can go on CLI tasks with a small, focused fine-tune.

Approach

• Fine-tuned TinyLlama-1.1B-Chat with LoRA on 150+ real-world CLI question-answer examples — 4.2M trainable parameters, 0.38% of the model.
• Trained in about 35 minutes on a Google Colab T4.
• Wrapped the model in an agent that plans step by step, extracts commands and runs them in dry-run mode with safety checks and full execution logs.
• Evaluated statically against the base model (BLEU, ROUGE-L) and dynamically on agent test cases.

Stack

• PyTorch
• Hugging Face
• LoRA
• TinyLlama