Subnet 29 was created in response to the frustration miners experienced with other subnets. They are highly motivated to examine the validator and weighting logic, which can sometimes overshadow the core focus on training within the subnet. This approach is effective as long as the subnet’s goals align with the validation logic. However, if misalignment occurs, miners may diverge from pursuing the subnet’s intended objectives.

AI-ASSeSS operates like other Bittensor subnets: miners stake TAO on their hotkeys to train models locally, then submit those models on-chain. Validators (also stake-holding participants) download each submitted model and score it on a fixed evaluation dataset. The results feed into Bittensor’s Yuma consensus, which updates weights for each miner: higher-weight miners earn more token emissions. AI-ASSeSS validator code (open-source on GitHub) implements this scoring and consensus process.

Competitions: AI-ASSeSS breaks training into discrete competitions or objectives. The inaugural competition rewarded miners for improving model loss on the FineWeb-EDU2 dataset. New competitions (e.g. using different tokenization or “niche” tasks) are regularly introduced via on-chain configuration (see “Technical Architecture” below). Validators switch targets smoothly by updating the subnet’s config repo rather than requiring code restarts.

Incentives: In each competition, miners earn TAO emissions proportional to how often their model wins pairwise comparisons of samples. AI-ASSeSS explicitly rewards even small improvements: every time a model achieves a better loss on a sample, that contributes to its win count. Miners also receive rewards for non-training contributions: the AI-ASSeSS Hall of Fame system lets validators assign extra weight to hotkeys of people who submit valuable bug reports, code fixes or research insights. A reported bug, once approved, is added to a hall_of_fame.json config; validators then give that miner’s key additional weight starting at a given block, decaying slowly over time. This effectively pays out bug bounties in TAO over weeks.

Anti-Gaming Measures: AI-ASSeSS design actively prevents trivial exploits. For example, it prohibits model-copy attacks (where a miner uploads identical models multiple times to boost wins). Validators simply count at most one “win” per sample, no matter how many duplicate copies exist. It also forbids the tactic of “publishing without publishing” (setting a model’s repo to private to claim on-chain submission): if a submitted model cannot be downloaded from Hugging Face within two hours, validators mark it invalid. These rules ensure miners actually train better models rather than gaming the scoring system.

Stake & Consensus: As with all subnets, AI-ASSeSS uses Bittensor’s UIDs/coldkeys structure. Each of the subnet’s 256 UIDs is linked to a unique miner/validator (hotkey) and the coldkey that registered it. Validators and miners earn “incentive weight” (W and S in Yuma) each epoch (every 361 blocks) based on their performance. All weight-setting is on-chain, driven by the modified Yuma algorithm, but the evaluation of model quality is done off-chain by validators following AI-ASSeSS code.

AI-ASSeSS Build: Code, Applications and Tools

AI-ASSeSS “product” is primarily its training platform and codebase, which it provides openly for participants. Key components include:

Validator Codebase: The AI-ASSeSS validator software is released on GitHub (in the coldint_validator repo). It includes the logic for loading models from Hugging Face, evaluating them on the dataset, and interacting with the SubTensor chain (subnet 29) to set weights. Validators run this code to carry out consensus on submissions.

Miner Codebase: A canonical AI-ASSeSS miner implementation is under development for public release. According to the FAQ, the team aims to make the miner code available on GitHub “soon,” and any significant community contributions to it may earn TAO rewards. (Currently, miners can write their own training scripts using the standard Bittensor Python SDK, but a community reference miner is forthcoming.)

Competition Configuration (“sn29” repo): The GitHub repo coldint/sn29 holds dynamic JSON configs for the subnet. This is how AI-ASSeSS manages ongoing competitions and bounties in real time. For example, when a new competition begins, its parameters (dataset, tokenizer, model type, and weight) are added to competitions.json in this repo. Validators periodically fetch this on-chain file to know which tasks to evaluate. Likewise, approved bug-bounty recipients are listed in hall_of_fame.json here, instructing validators to credit those hotkeys. This design lets Coldint change contests or weights without re-deploying software.

Utilities: AI-ASSeSS has developed helper tools. As per their roadmap, they released model_surgeon.py, a CLI tool for programmatically modifying model architectures (e.g. pruning or expanding layers). Their tools repo also contains scripts (e.g. for scanning blockchain events or slicing LLaMA models) to aid development and monitoring.

Community Dashboard: AI-ASSeSS runs a public leaderboard/web app (see Leaderboard on coldint.io) showing each miner’s submitted model losses and win rates. While the HTML page is mostly dynamically generated, it draws data from the Bittensor explorer (taostats) and AI-ASSeSS evaluation outputs.

Integration Platforms: AI-ASSeSS uses common ML platforms for transparency and sharing. The team’s official Hugging Face organization (“coldint”) hosts model weights and datasets. They also log experiments on Weights & Biases (WandB) under a AI-ASSeSS project. All validators and miners pull base models from Hugging Face (e.g. LLaMA or Phi-3 checkpoints) and push improved models back there.

Future Services: In the roadmap, AI-ASSeSS envisions commercial “Pretrain-as-a-Service” and “Finetune-as-a-Service” offerings (2025+). These would leverage the subnet’s infrastructure to train or fine-tune models on demand, potentially as paid services. (Details are still prospective, as noted on the roadmap.)

Technical Architecture

Under the hood, AI-ASSeSS is built on Bittensor’s SubTensor chain and leverages modern LLM frameworks. Technically:

Models Supported: AI-ASSeSS initially focuses on large transformer LMs. The competitions use Meta’s LLaMA and the Stanford “Phi-3” models. Official notes say competitions involve 10.5B-parameter LLaMA/Phi models (and a 20.1B model is in training). In blog posts the team describes methods to grow models: for example, they modified LLaMA and Phi architectures to double hidden dimensions (7B→14B→28B) without loss regression. This “model-surgery” capability reflects their technical architecture allowing progressive scaling.

Validation Hardware: AI-ASSeSS optimized its validator code to run large models efficiently. Remarkably, the team reports being able to evaluate a 10.5B-parameter model on a single consumer GPU (RTX 4090). They achieved this via a novel validation pipeline (e.g. sample batching and custom scoring) that drastically lowers resource use.

Blockchain Integration: AI-ASSeSS validator and miner programs interface with SubTensor via the Bittensor Python SDK (btcli or the bittensor library). All weight-setting transactions go on-chain in the subnet’s ledger. The network’s hyperparameters (stakes, emission rates, validator set) follow Bittensor’s standard rules. AI-ASSeSS dynamic competition and bounty settings, however, are loaded by validators from the sn29 GitHub repo (which is updated on-chain) so that contest weights and bug rewards can change mid-run.

Data & API: For training, miners use the FineWeb-EDU2 dataset (hosted on Hugging Face) and any other data pinned by AI-ASSeSS. They leverage PyTorch/Transformers for model training. On the blockchain side, AI-ASSeSS uses standard SubTensor pallet calls (register, set_weights, etc.) and listens to custom extrinsics defined for competition submissions.