Skip to content

LoRA fine-tuning

LoRA fine-tuning on the cheapest GPU available

When a tuned prompt isn't enough, train weights. Text LoRA trains a real adapter on an open-weight model from your input/expected rows and reports real train and eval loss; Audio LoRA trains one on ACE-Step 1.5 from a ZIP of your audio. Kojen rents the GPU, runs the job and stops the machine when it's done.

Get started free

Free to start · Pay as you go · No card needed

What it does

Text LoRA

PEFT + TRL supervised fine-tuning on Qwen3-8B or Llama-3.1-8B-Instruct, from a dataset with at least 20 training rows. Set epochs (up to 10), rank (up to 64), learning rate and sequence length. The test split is never exported, so evaluating against it stays clean.

Audio LoRA

Upload a ZIP of at least 4 audio files, with optional captions, and confirm you have the rights to them. Kojen trains an ACE-Step 1.5 adapter and plays base and tuned samples side by side.

Cheapest GPU, with fallback

Runs train on Modal, Runpod, Vast.ai or Lambda. "Cheapest" (the default) takes the lowest live price with a GPU in stock and falls back to the next offer if a launch fails. Llama-3.1-8B-Instruct trains on Modal only.

Pay for the time used

A run reserves its worst-case cost in GPU credits at the GPU's live price, locked when it starts, and is charged only for the GPU time it used, including when it fails or you cancel it.

How it works

  1. 1Upload a dataset with input and expected columns (or a ZIP of audio).
  2. 2On Pipelines → New pipeline, choose Text LoRA or Audio LoRA, the base model and settings.
  3. 3Pick a GPU provider or leave it on cheapest; buy credits if the run needs more.
  4. 4Follow the loss as it trains; the adapter is saved as a draft model version.

Limits, plainly

  • Early access: GPU training is on for enabled accounts, one run at a time, up to 4 hours a run.
  • Adapters are saved as draft versions and can't serve evaluations or deployments yet.
  • Full fine-tuning and RLHF for text models are previews: they train no weights and report no metrics.

Questions

What is LoRA fine-tuning?
Low-Rank Adaptation freezes the base model and trains small adapter matrices alongside it, so fine-tuning an 8B model fits on one GPU and the result is a file of a few hundred megabytes rather than a full copy of the model.
How much GPU memory does it need?
It depends on the model, rank, sequence length and batch size. The fine-tuning VRAM calculator estimates it for full fine-tuning, LoRA and QLoRA.
Can I serve the adapter from Kojen?
Not yet. Adapters are saved as draft model versions with their loss metrics; deployments currently serve hosted models with tuned prompts.

Read more

More of Kojen

Get started free

Free to start · Pay as you go · No card needed