Flower Labs collaboration benchmark
EdgeFlowerTune
Deployment-oriented federated LLM fine-tuning under realistic edge-system constraints.
Meet EdgeFlowerTune · Team introduction
This recorded introduction illustrates the original research setup. See the current participant guide for this release's submission definitions.
Flower Labs collaboration benchmark
EdgeFlowerTune
Deployment-oriented federated LLM fine-tuning under realistic edge-system constraints.
Final score
Total Ranking
Higher score is better. Final score is the weighted average of Protocol A/B/C scores, with default protocol weights 1:1:1.
| Rank | Method | Final Score | Protocol A | Protocol B | Protocol C | Weight Total |
|---|
Score summary
Overall Ranking by Protocol
Higher scores reflect better task-level ranks.
Check out the Protocol Boards
Benchmark Pages
Model Quality
Best evaluated checkpoint accuracy and loss, with calibrated system costs recorded separately.
Protocol BCost to Target
Wall-clock time, communication, energy, and peak memory required to hit target accuracy.
Protocol CRobustness
Robustness across seven datasets and all three backbone models.
PlatformReal Devices
Heterogeneous Android and Jetson edge clients used by the benchmark.
Per-model scores
Score Details by Model
| Protocol | Rank | Method | Base Model | Score | Feasibility | Note |
|---|
Score summary
Overall Ranking
Compare the task-level results below.
Protocol A
Model Quality
Models: Qwen2.5-0.5B, Gemma 3-270M, Gemma 3-1B. Datasets: BoolQ, QNLI, PIQA, HellaSwag, SocialIQA, ARC-Easy, WinoGrande. Results are grouped by model and dataset.
| Model | Task | Dataset | Method | Loss | Accuracy | Status |
|---|
Protocol B
Cost to Target
Models: Qwen2.5-0.5B, Gemma 3-270M, Gemma 3-1B. Datasets: BoolQ, QNLI, PIQA, HellaSwag, SocialIQA, ARC-Easy, WinoGrande. Results are grouped by model and dataset.
| Model | Task | Dataset | Target Acc | Method | Time h | Comm MB | Energy kJ | Memory MB | Status |
|---|
Protocol C
Robustness
Models: Qwen2.5-0.5B, Gemma 3-270M, Gemma 3-1B. Datasets: BoolQ, QNLI, PIQA, HellaSwag, SocialIQA, ARC-Easy, WinoGrande. Results are grouped by model and dataset.
| Model | Scenario | Condition | Dataset | Method | Accuracy % / Δ pp | Time h / Δ h | Comm MB / Δ MB | Energy kJ / Δ kJ | Memory MB / Δ MB |
|---|
Platform
Real Edge Devices
| Order | Device | Processor | Memory |
|---|
Submission pipeline
Submission
Checking the submission mode...
The upload form stays disabled until the server configuration is available.
Backend jobs
Recent Submissions
| Submission | Protocol | Stage | Status | Updated | Result |
|---|
Methods on the leaderboard
Submitted Methods
Protocol A: quality · B: cost to target · C: robustness. Scores use the leaderboard's 0–10 scale; farther from the center is better. Missing scores remain unfilled. Benchmark reference methods are labeled below.
Other Leaderboards