Fine-Tuning Qwen3-8B for Medical Reasoning on E2E Networks A100 GPU

Over the last few years, open-weight large language models have transitioned from mere text generation systems into versatile reasoning engines that can simulate analytical thinking and pattern recognition across varied contexts. Despite this progress, most LLMs are trained for breadth, not depth. They perform well on generic instructions but lack the structured reasoning patterns required in specialized fields like clinical diagnostics. When prompted with complex cases, they often jump directly to answers through statistical association rather than deduction, bypassing intermediate logic.

In medicine, however, causal reasoning is everything. The difference between correlation and mechanism can define clinical accuracy. A general-purpose model may predict that "excessive thirst" indicates diabetes, but it fails to connect the metabolic sequence — hyperglycemia → osmotic diuresis → dehydration → thirst. Without this chain-of-thought, explanations sound plausible yet remain medically superficial. This limitation underscores a central challenge in LLM engineering today: training models not just to generate, but to reason faithfully within constrained scientific logic.

Our goal in this experiment is to close that gap by aligning Qwen3-8B with domain-grounded reasoning behaviors. We accomplish this through supervised fine-tuning (SFT) on FreedomIntelligence/medical-o1-reasoning-SFT, a dataset explicitly designed for reasoning enhancement. Each data point includes:

1. A Question — the diagnostic prompt.
2. A Complex_CoT — a human-annotated chain-of-thought containing structured intermediate reasoning.
3. A Response — the final diagnosis or explanation. This format lets the model learn not only answers but the causal scaffolding leading to them — a process that directly improves trust and interpretability.

From a systems-engineering standpoint, training such large models for domain adaptation demands careful trade-offs between resource efficiency, numerical precision, and model stability. We approach this using 4-bit quantization via BitsAndBytes to compress model weights by 75%, preserving performance while making the workflow viable on a single A100 GPU (80 GB).

Next, we apply QLoRA (Quantized Low-Rank Adaptation), where small trainable adapters capture reasoning deltas without updating the full model. This approach ensures that fine-tuning remains computationally light yet behaviorally impactful — especially for logical reasoning domains where coherence, not creativity, is the optimization target.

Throughout this workflow, we emphasize reproducibility and precision tracking. All experiments are conducted on E2E Networks Cloud, leveraging persistent volumes and controlled CUDA environments to eliminate transient variability in runtime or caching. Each notebook corresponds to a single reproducible engineering phase — baseline testing, quantization, QLoRA fine-tuning, adapter evaluation, and inference benchmarking. By structuring it this way, the entire process can be re-run deterministically by any engineer without reconfiguration.

Ultimately, this project serves two purposes:

1. to demonstrate that open-weight models like Qwen3-8B can be tuned for medical reasoning on accessible hardware, and
2. to highlight that fine-grained reasoning alignment is achievable without massive GPU clusters or full-model retraining.

This is a practical engineering proof-of-concept that bridges research methodology and applied deployment, showing how lightweight reasoning alignment can lead to measurable gains in accuracy and interpretability within domain-specific contexts.

Key Takeaways

• Fine-tune Qwen3-8B in ~15–20 hours on A100 GPU
• E2E Networks provides A100 access at ₹220/hour
• 75% VRAM reduction with QLoRA + 4-bit quantization

Before training begins, one of the most critical architectural decisions lies not in the model, but in the dataset — what kind of reasoning signal it carries, and how that signal interacts with the model's internal representations. For reasoning-centric fine-tuning, the goal is not exposure to more text; it's exposure to better-structured logic. If the data simply maps questions to answers, the model learns shortcut correlations. But when the data explicitly encodes the intermediate reasoning — the why and how — it teaches the model to internalize causal pathways rather than surface patterns.

For this project, the dataset effectively becomes the curriculum that shapes cognitive behavior inside Qwen3-8B. The objective isn't to teach new vocabulary but to bias attention flows toward multi-step inference, error correction, and structured decision paths. This is the foundation of "domain reasoning alignment": ensuring that when the model reasons, it does so in a way that mirrors real diagnostic logic rather than probabilistic guessing.

A medical reasoning dataset must therefore satisfy three design constraints:

1. It should contain explicit causal traces — not just final outcomes.
2. It must represent diverse pathophysiological cases to avoid overfitting to a single specialty.
3. Its text distribution should maintain technical language density, since linguistic complexity directly affects token utilization and reasoning depth.

The FreedomIntelligence/medical-o1-reasoning-SFT dataset meets these conditions precisely. Each record includes a Question, a detailed Complex_CoT (chain-of-thought), and a Response, allowing the model to learn end-to-end diagnostic reasoning flows. Unlike generic medical QA datasets, this one treats the "thinking" process as the supervision target — transforming fine-tuning into a controlled reasoning calibration task rather than rote memorization.

From an engineering viewpoint, this choice simplifies optimization downstream. Models trained on explicit reasoning traces converge faster and exhibit higher gradient stability, since intermediate steps provide natural regularization. They also demonstrate better transfer to unseen prompts because their latent activations encode general reasoning patterns instead of narrow answers.

In the next section, we'll dissect the dataset's schema, preprocessing pipeline, and text formatting logic — showing how these elements translate into a reproducible supervised fine-tuning setup for QLoRA.

Step 1 — Understanding and Structuring the Dataset

The dataset serves as the operational foundation of this fine-tuning workflow. We use FreedomIntelligence/medical-o1-reasoning-SFT, a supervised dataset specifically curated for chain-of-thought learning in biomedical contexts. Rather than providing direct Q-A pairs, it embeds explicit reasoning traces that teach the model how to arrive at an answer — a property essential for medical decision logic.

---

1 Dataset Overview

• Source: Hugging Face → FreedomIntelligence/medical-o1-reasoning-SFT (en)
• Fields: Question, Complex_CoT, Response
• Scale: ≈ 19 k records of multi-specialty clinical reasoning
• Format Intent: Each sample represents a full reasoning chain → diagnostic conclusion sequence

The Complex_CoT field encodes intermediate thought steps (cause → effect → inference). During training, these become the supervision target for reasoning alignment.

---

2 Why This Dataset Works for QLoRA

From an engineering standpoint, this dataset has structural advantages for quantized fine-tuning:

1. Stable gradient behavior — Each reasoning trace adds intermediate supervision, which smooths the loss landscape and minimizes exploding updates in low-precision training.
2. Causal representation learning — Explicit step-wise text encourages the model to form internal dependencies between medical variables instead of memorizing keyword mappings.
3. Cross-domain coverage — Includes prompts across neurology, endocrinology, nephrology, cardiology, and infectious disease, allowing generalization without catastrophic forgetting.
4. Balanced text length — Typical record length falls between 250–800 tokens, a sweet-spot for 4-bit memory constraints.

This makes it ideal for controlled fine-tuning on limited-VRAM hardware while still producing reasoning-dense gradients.

---

3 Schema Inspection and Verification

Before preprocessing, verify schema integrity and feature consistency:

Expected output:

If the loader raises ValueError: Config name is missing, specify "en" explicitly.

---

4 Formatting for Supervised Fine-Tuning

To align with QLoRA's input–target expectations, concatenate fields into a single structured text block. This ensures reasoning traces are learned as part of the forward pass.

Create a compact training subset for testing, then scale later:

---

5 Sanity Checks and Token Length Estimation

Large reasoning traces can cause sequence truncation. Estimate median token length to calibrate max_seq_length before training:

This value guides your training configuration for packed sequences or truncation thresholds.

---

6 Engineering Notes

• Always use the "en" config to avoid loader ambiguity.
• Store the preprocessed subset inside /home/jovyan/ for persistence between sessions.
• Validate random samples after mapping — mis-aligned tags (<|think|>) can silently degrade reasoning accuracy.
• Maintain JSONL formatting; it supports streaming for large-scale continuation runs.

---

At this stage, the dataset is fully verified, formatted, and ready for quantized model loading.

The next step focuses on infrastructure setup and environment provisioning for QLoRA fine-tuning.

---

Step 2 — Infrastructure Setup

Running large-scale models like Qwen3-8B requires a stable GPU environment. We use E2E Networks Cloud because it provides high-memory NVIDIA A100 GPUs, persistent storage, and direct JupyterLab access — ideal for reproducible fine-tuning workflows.We run everything in E2E's managed Jupyter with an A100 80GB instance. Model caches persist under /home/jovyan.

---

1. Launch your E2E instance

Steps followed:

1. Log in to TIR E2E Networks.
2. Click Create Project → select your project name and region.
3. Under "Images," choose the PyTorch base image (it includes CUDA and Python pre-installed).
4. Under "GPU," pick NVIDIA A100 80GB.
5. Set storage to 400 GB (recommended for large model checkpoints).
6. Add your SSH key (optional, for external access).
7. Click Launch Instance to start your workspace.

This configuration ensures enough VRAM for 8B-parameter model fine-tuning and a buffer for quantized adapters and cached tensors.

---

2. Verify instance status

Once launched, you'll see the instance dashboard showing:

• Instance ID and public IP
• GPU type (A100 80GB)
• Attached volume size
• State: "Running"

You can now connect directly to the JupyterLab environment via your browser.

Instance configuration with A100 GPU, 400 GB storage, and SSH setup.

---

3. Access JupyterLab

When your instance is active:

1. Click Open JupyterLab on the E2E dashboard.

2. Navigate to the workspace folder:

   /home/jovyan/

   This directory persists between restarts — store models and checkpoints here.

3. You should see your notebooks (.ipynb), datasets, and cache folders.

---

4. Basic environment verification

Before running anything, confirm Python and CUDA are ready:

Example output:

Python 3.12.3 pip 24.0 Driver Version: 535.154.05 CUDA Version: 12.2 GPU: NVIDIA A100 80GB PCIe

If CUDA isn't detected, reboot the instance once from the dashboard.

---

5. Install runtime dependencies

Start your notebook and install the minimal required libraries:

You'll see confirmation like:

Successfully installed accelerate-0.34.2 ...

---

Notes for reproducibility

• Always store outputs, checkpoints, and cached models inside /home/jovyan — this path survives restarts.
• Avoid /tmp or /root, as those reset when the instance reboots.
• You can snapshot your volume from the E2E dashboard to save compute time for future experiments.

Cost Analysis: E2E Networks India Pricing

Before proceeding with training, here's a breakdown of GPU costs on E2E Networks:

Estimated Training Time: 15-20 hours for 3 epochs on 5K samples Cost for This Tutorial: Approximately ₹3,300 - ₹4,400 (on A100 80GB)

Pricing subject to change. Check E2E Networks pricing page for current rates.

---

Next step: with the infrastructure ready, we'll load and quantize the Qwen3-8B baseline model for reasoning evaluation.

Step 3 — Baseline Model Loading and Reasoning Test

Before quantization or fine-tuning, verify that the Qwen3-8B base model runs correctly and produces structured reasoning outputs. This serves as the control reference for post-training comparison.

---

1. Load Dependencies

accelerate manages device mapping automatically during model initialization.

---

2. Load the Baseline Model

We load the full-precision Qwen3-8B checkpoint directly from Hugging Face:

This version uses standard precision — no quantization applied yet.

---

3. Test the Model with a Baseline Prompt

To see how well the model reasons before fine-tuning, we run a clinical question under "thinking mode."

This gives you the baseline diagnostic reasoning that will be improved after fine-tuning.

---

Baseline output from Qwen3-8B before quantization or fine-tuning.

---

Step 4 — Quantization for Efficient Fine-Tuning

Qwen3-8B in full precision can saturate even high-memory GPUs. Quantizing to 4-bit keeps it stable on a single A100 (80 GB) while maintaining reasoning precision and stability close to FP16 To enable efficient fine-tuning on a single A100 GPU (80 GB VRAM), we first quantize the model to 4-bit precision.

---

1. Why Quantization?

Quantization compresses model weights from 16-bit floating point to 4-bit representations. This lowers the memory footprint and speeds up computation with minimal accuracy loss.

Benefits:

• Reduces memory consumption by up to 75%
• Keeps the model trainable for QLoRA fine-tuning
• Works seamlessly with BitsAndBytes (bnb) integration in Transformers

---

2. Run Quantization Code

Open a new Jupyter cell and execute the following:

---

3. Verify the Quantized Folder

Once quantization finishes, confirm that the files are saved properly:

You should see entries like:

config.json model.safetensors tokenizer.json

---

Successfully quantized model stored under /home/jovyan/qwen3_8b_quantized.

---

4. Proceed to Fine-Tuning

We now have a lightweight 4-bit quantized model ready for QLoRA fine-tuning. In the next step, we'll connect this checkpoint and train it on the medical reasoning dataset to enhance logical depth and clarity.

Step 5 — Fine-Tuning Qwen3-8B with QLoRA

With the 4-bit quantized checkpoint ready, we fine-tune the model using QLoRA — a method that applies low-rank adapter layers on top of the frozen quantized backbone. This approach retains full model expressiveness while keeping GPU memory usage manageable.

---

1. Why QLoRA

QLoRA (Quantized Low-Rank Adaptation) allows parameter-efficient training where only adapter weights update while the main model stays frozen. This design enables large-scale fine-tuning even on a single A100 GPU.

Key advantages:

• Keeps training feasible on a single high-memory GPU.
• Reduces VRAM load by over 70%.
• Maintains reasoning fidelity close to full precision.
• Outputs compact, reusable adapter files.

---

2. Install Dependencies

Install all fine-tuning libraries and ensure versions are recent:

---

3. Configure and Run Training

We use the quantized checkpoint (./qwen3_8b_quantized) as the base model and train adapters on the medical reasoning dataset.

---

4. Save Adapter Checkpoint

When training completes, export the adapter weights:

Files generated:

adapter_config.json adapter_model.safetensors README.md

---

Training progress showing stable convergence across fine-tuning steps.

---

5. Technical Summary

• The quantized backbone remains frozen in 4-bit precision.
• The LoRA adapter captures all domain reasoning updates.
• Total footprint ≈ 500 MB — a fraction of the full model size.

---

Next: load the adapter and run post-training inference to benchmark reasoning improvements.

Step 6 — Load Adapter and Run Post-Training Inference

With the LoRA adapter saved, we now reload the quantized base model and attach the adapter to evaluate post-training reasoning improvements. This validates that domain alignment worked as intended and that adapter weights integrate cleanly with the base.

---

1. Load Quantized Base and Attach LoRA Adapter

---

2. Post-Training Inference on Baseline Prompt

Use the same prompt from the baseline test to visualize improvements in structured reasoning.

---

This step demonstrates the adapter's effect on reasoning coherence. Use the identical prompt as the baseline to highlight logical expansion and improved chain-of-thought consistency.

---

Notes

• Use the same baseline prompt for consistent evaluation.
• If you hit a cpu vs cuda mismatch, ensure .to(model.device) is applied to tokenized inputs (included above).
• Both model and adapter must share the same quantization precision (bfloat16 in this case).

---

Post-training reasoning output for the same clinical prompt, showing improved causal structure.

Step 7 — Evaluation Methodology

With both baseline and fine-tuned models ready, evaluation focuses on verifying measurable gains in reasoning alignment. The comparison uses unseen clinical prompts drawn from neurology, nephrology, endocrinology, respiratory, and systemic domains.

---

1. Evaluation Objectives

Three axes guide the assessment:

• Reasoning depth — Does the model build intermediate causal logic before reaching conclusions?
• Factual correctness — Are explanations medically sound and internally consistent?
• Hallucination control — Does the model avoid fabricated or off-topic reasoning chains?

These indicators distinguish genuine reasoning improvement from verbosity.

---

2. Evaluation Code

---

3. Evaluation Output

Screenshots below show model behavior on previously unseen prompts.

Baseline Output — surface reasoning, limited causal expansion.

Fine-Tuned Output — multi-step clinical reasoning with explicit causal links.

---

4. Observations Summary

---

The contrast confirms that adapter-based fine-tuning enhanced reasoning organization and factual retention without inflating verbosity. The next step consolidates these results into quantitative indicators and memory metrics.

Step 8 — Observations & Discussion

Post-evaluation, both qualitative and quantitative analyses confirm that the fine-tuned model demonstrates measurable reasoning gains over the baseline. The following observations consolidate behavior across multiple unseen clinical scenarios.

---

1. Reasoning Depth

The fine-tuned Qwen3-8B (Medical Reasoning) consistently produced deeper, multi-step reasoning sequences. These chains mirrored the Complex_CoT structure of the dataset — moving beyond symptom listing toward causal logic. Where the baseline relied on pattern recall, the tuned model linked physiological mechanisms (e.g., "insulin deficiency → lipolysis → ketogenesis") before producing conclusions.

---

2. Factual Correctness

Accuracy improvements were clear in endocrine, renal, and metabolic prompts. The model's responses reflected better internal medical consistency, particularly in connecting diagnostic pathways. Minor factual drift remained on rare or ambiguous cases, but average correctness was notably higher compared to baseline runs.

---

3. Language and Clarity

The fine-tuned model transitioned from short declarative statements to structured clinical reasoning. Many responses initiated with context-aware cues ("Let's reason through this…") before explaining intermediate logic. This indicates that the QLoRA procedure enhanced coherence and logical sequencing without impacting fluency.

---

4. Quantitative Indicators

Approximate metrics (sampled from 10 evaluation prompts):

(Values estimated from manual review and token-length statistics.)

---

5. Memory and Performance

The quantized 4-bit model maintained ~38 GB VRAM utilization on the A100 80 GB GPU during inference. This allowed full-sequence evaluations (≈8 k tokens) without activation offloading or gradient checkpointing.

---

Summary: QLoRA-based fine-tuning improved reasoning coherence, factual precision, and causal trace alignment — all while retaining an efficient 4-bit footprint. These findings confirm that quantized adapter tuning is a practical, low-cost strategy for domain-aligned reasoning models.

---

Next step: we document reproducibility measures and scaling considerations for expanding beyond the current subset.

Step 9 — Reproducibility & Next Steps

A reproducible setup ensures that this workflow can be re-run on E2E's GPU infrastructure without model re-downloads or data loss between sessions. Proper directory structure and checkpoint persistence are key for iterative fine-tuning and evaluation.

---

1. Persisting Models and Caches

E2E Jupyter environments retain all data stored under /home/jovyan/ after restarts. Always place models, adapters, and logs in this directory to preserve state:

This layout guarantees reproducibility and isolates checkpoints from transient runtime paths such as /tmp or /root.

---

2. Checkpoint Reuse

To resume fine-tuning or evaluation:

This attaches the saved adapter directly to the quantized base, restoring the model to its fine-tuned state.

---

3. Scaling for Extended Training

For larger runs, extend dataset usage beyond the initial subset and raise max_steps or num_train_epochs. The existing setup scales efficiently to 5,000–20,000 examples without exceeding the 80 GB VRAM limit.

Monitoring GPU utilization via nvidia-smi helps validate throughput consistency.

---

4. Version Control and Documentation

Each experiment snapshot should include:

• A requirements.txt for dependencies
• Model logs and training hyperparameters
• Commit tags in Git or DVC for lineage tracking

---

Next step: proceed to Step 10 — Conclusion, summarizing the engineering outcomes and deployment readiness.

---

Conclusion

This project established a complete, end-to-end workflow for domain reasoning enhancement using Qwen3-8B, 4-bit quantization, and QLoRA fine-tuning on the FreedomIntelligence/medical-o1-reasoning-SFT dataset.

Quantizing the base model before training reduced GPU memory usage by nearly 50%, while maintaining reasoning precision and stability. Adapter-based fine-tuning enabled alignment on a single A100 (80 GB) instance without the need for gradient checkpointing or model offloading.

The fine-tuned adapter consistently improved clarity, causal reasoning, and factual alignment on unseen clinical prompts — producing structured diagnostic chains absent in the baseline. Evaluation confirmed higher reasoning coherence and a significant reduction in hallucination frequency.

This implementation provides a practical blueprint for engineers aiming to:

• Fine-tune open-weight LLMs efficiently on modest GPU infrastructure
• Adapt models for specific reasoning-intensive domains
• Maintain full reproducibility using persistent cloud storage and lightweight adapters

The combination of quantization + LoRA adapters remains one of the most effective approaches for real-world LLM alignment — combining computational efficiency with transparent reasoning.

---

End of workflow. Future iterations can extend to multi-domain reasoning datasets or integrate retrieval augmentation for richer factual grounding.