Wagner Solutions AI — Santiago, Chile
*Corresponding author: sebastian@wagnersolutionsai.com †Conceptual and computational co-design
Hyperparameter optimization (HPO) and neural architecture search (NAS) remain sample-inefficient despite advances in Bayesian optimization (BO) and, more recently, LLM-based optimizers such as OPRO. We introduce Myco-Opt, a framework that uses graph-theoretic properties of mycorrhizal networks — resource-transport networks shaped by millions of years of evolution under uncertainty — as biological priors to constrain the search space of LLM-driven optimizers. Mycorrhizal networks exhibit well-characterized structural features (modular guild organization, scale-free degree distributions, small-world connectivity, and adaptive resource allocation) that map naturally onto the design space of transformer-based language models (mixture-of-experts routing, layer width profiles, attention patterns, depth, and curriculum scheduling).
We build mycorrhizal interaction graphs from GlobalFungi (600M+ fungal observations) and MycoDB (experimental inoculation effects), encode them in a graph database, and extract network metrics that parameterize a biologically-grounded prior over hyperparameter space. An LLM optimizer (OPRO-style) proposes candidate configurations within this constrained space, receives training feedback, and iterates in an evolutionary loop.
On a causal language modeling proxy task (7–30M parameters, Chilean Spanish corpus), we benchmark Myco-Opt against random search, Hyperband, Optuna (BO), and vanilla OPRO. We hypothesize that biological priors reduce the evaluation budget required to reach a target loss by up to 50% relative to agnostic methods, while yielding structurally distinct, more interpretable configurations. We release all code, data pipelines, and the graph-to-hyperparameter mapping as open source.
Training large language models (LLMs) is expensive: a single run of a frontier-scale model can cost millions of dollars in compute. Within this budget, the search for good hyperparameters — depth, number of experts, learning rate, routing topology, dropout — remains a largely trial-and-error process. Modern HPO methods reduce waste but are still sample-inefficient: they require many costly evaluations to converge.
In parallel, a new class of optimizers treats LLMs themselves as optimizers (OPRO; Large Language Models as Optimizers, DeepMind 2023). The optimizer LLM proposes candidate configurations, receives evaluation feedback, and iterates. OPRO and its relatives (FunSearch) have shown that LLMs can discover competitive solutions — but they explore the search space agnostically, without structural priors.
This paper introduces Myco-Opt, which injects a biological prior into the search: the topology of mycorrhizal networks. Mycorrhizal networks are resource-transport systems formed by millions of years of evolution under uncertainty. Their structural properties — modular guilds, scale-free degree distributions, small-world connectivity, and adaptive resource allocation — map naturally onto the design space of transformers. We use these properties to constrain the search space of an LLM-driven optimizer, reducing blind exploration.
Bayesian Optimization (BO) builds a surrogate of the objective and queries where improvement is expected. Hyperband uses adaptive resource allocation (successive halving). Both are strong baselines but degrade under small evaluation budgets and high-dimensional search spaces (Bergstra et al., 2011; Li et al., 2017).
OPRO (Yang et al., 2023) frames optimization as a prompt: the optimizer LLM proposes solutions from natural-language descriptions of past trials. FunSearch (Romera-Paredes et al., 2024) applies a similar loop to program synthesis. These methods are priors-free: every region of the search space is a priori equally plausible.
The slime mold Physarum polycephalum reconstructs near-optimal transport networks (Nakagaki et al., 2000; Tero et al., 2010, published in Science), inspiring a literature on organism-to-algorithm transfer. Myco-Opt extends this tradition to hyperparameter optimization of LLMs, using graph-theoretic metrics of real fungal datasets rather than abstract dynamics.
A mycorrhizal network is a graph \(G = (V, E)\) where \(V\) contains plants and fungi, and edges represent co-occurrence or resource transfer. Three structural properties are of interest:
Each property has a standard quantitative measure, and each maps to a transformer design knob (Section 3.4).
We construct three graphs from open datasets:
Graphs are stored in a graph database (MillenniumDB, RDF) and queried via SPARQL to extract the metrics of Section 3.3.
We compute five classical metrics:
where \(A\) is the adjacency matrix, \(k_i\) the degree of node \(i\), \(c_i\) its community label, \(t_i\) the number of triangles at \(i\), and \(d(i,j)\) the shortest-path distance.
Each metric constrains one (or more) transformer design knobs. The mapping is a search prior, not a fixed value — the evolutionary loop (Section 3.5) corrects it:
| Network metric 🍄 | Ecological meaning | LLM hyperparameter 🧠 | Example mapping |
|---|---|---|---|
| Modularity \(Q\) | Specialized fungal guilds | num_experts (MoE) | \(\text{round}(6/Q)\) |
| Degree exponent \(\gamma\) | Hubs concentrate flow | top_k routing | \(\text{clamp}(\text{round}(\gamma),1,8)\) |
| Small-world \(\sigma\) | Local signal + global shortcuts | num_heads | \(\text{round}(\sigma \cdot 4)\) |
| Mean path \(L\) | How far carbon travels | num_layers | \(\text{round}(L \cdot 1.5)\) |
| Clustering \(C\) | Local redundancy | dropout | \(\text{clamp}(1-C,\,0.1,\,0.3)\) |
| Resilience to removal | Survives node failure | regularization / sparsity | — |
| Carbon-flow dynamics | Prioritizes by need | curriculum / LR schedule | — |
⚠️ These mappings are search priors: they restrict the search space toward biologically plausible regions. They do not dictate final values; the evolutionary loop corrects them.
Differences vs. vanilla OPRO: (1) biological prior restricts the search space; (2) mycelial memory: the optimizer records why configurations worked; (3) fungal curriculum: training-data ordering follows the network's resource-flow dynamics.
Before scaling to full LLM training, we validated the empirical foundation of the pipeline using dashAI (U. de Chile / CENIA): an open-source, local-first machine-learning workbench with a schema-driven UI. This validates the claim that network structure is predictable from node-level features — the basis of the surrogate model used inside the Myco-Opt loop.
We generated 48 synthetic mycorrhizal-like networks (Barabási-Albert, Watts-Strogatz, stochastic block model, and hybrid BA+SBM) with biologically realistic topologies. For each node we computed 21 structural features (degree, clustering, betweenness, closeness, eigenvector centrality, PageRank, core number, community size, graph-level metrics, …).
Target: \(\text{es\_critico}\) — whether a node is in the top 15% most damaging to network connectivity if removed (relative drop of the giant component):
Dataset: 5,000–8,991 rows · 21 features · 3 model sessions · 7 runs. All experiments ran on a 4-core CPU VPS (Oracle Cloud, 23 GB RAM) — no GPU required, highlighting dashAI's local-first design.
| # | Dataset | Model | Tuning | F1 (test) | Precision | Recall | ROCAUC |
|---|---|---|---|---|---|---|---|
| 2 | clean | RandomForest | fixed | 1.000 | 1.000 | 1.000 | 1.000 |
| 3a | clean | GradientBoosting | Optuna (10) | 1.000 | 1.000 | 1.000 | 1.000 |
| 3b | clean | HistGradientBoosting | Optuna (10) | 1.000 | 1.000 | 1.000 | 1.000 |
| 5 | noisy | RandomForest | baseline | 0.686 | 0.891 | 0.558 | 0.772 |
| 6 | noisy | GradientBoosting | Optuna (10) | 0.668 | 0.854 | 0.548 | 0.762 |
| 7 | noisy | HistGradientBoosting | Optuna (10) | 0.678 | 0.881 | 0.551 | 0.772 |
Noisy dataset: Gaussian noise (8–20% of feature std) added to all numerical features and 12% of labels flipped, simulating real-world measurement error.
dashAI was used end-to-end: dataset upload (UI), model sessions, Optuna integration, and metric collection. The schema-driven UI (forms generated from Pydantic schemas) made 7 experimental runs reproducible in minutes. This validates that the surrogate/meta-model component of Myco-Opt can be trained without writing code — democratizing the experimentation pipeline.
| Component | Value |
|---|---|
| Proxy task | Causal LM, 7–30M params, Chilean Spanish corpus (10–50M tokens) |
| Budget | 4–8 × H100-equivalent GPU-days |
| Methods | Random Search · Hyperband · Optuna (BO) · OPRO vanilla · Myco-Opt |
| Runs per method | 30 (with variance reporting) |
| Metrics | final loss · budget-to-target · run variance · structural interpretability |
| Ablation | Myco-Opt ± biological prior (does the prior add value over OPRO?) |
Hypotheses: