With the amount of new subnets being added it can be hard to get up to date information across all subnets, so data may be slightly out of date from time to time

Subnet 29

Coldint

Emissions
Value
Recycled
Value
Recycled (24h)
Value
Registration Cost
Value
Active Validators
Value
Active Miners
Value
Active Dual Miners/Validators
Value

ABOUT

What exactly does it do?

The Coldint subnet is dedicated to advancing collaborative, distributed model training and research, while fostering the exchange of innovative ideas on model architecture, training, and evaluation. It originated as a fork of Subnet 9 (pretraining), which was relatively static and lacked incentives for sharing incremental improvements.

Their mission is to enhance the collective training efforts within the Bittensor community by incentivizing the exchange of models, knowledge, insights, and code.

The Coldint subnet is dedicated to advancing collaborative, distributed model training and research, while fostering the exchange of innovative ideas on model architecture, training, and evaluation. It originated as a fork of Subnet 9 (pretraining), which was relatively static and lacked incentives for sharing incremental improvements.

Their mission is to enhance the collective training efforts within the Bittensor community by incentivizing the exchange of models, knowledge, insights, and code.

PURPOSE

What exactly is the 'product/build'?

Subnet 29 was created in response to the frustration miners experienced with other subnets. They are highly motivated to examine the validator and weighting logic, which can sometimes overshadow the core focus on training within the subnet. This approach is effective as long as the subnet’s goals align with the validation logic. However, if misalignment occurs, miners may diverge from pursuing the subnet’s intended objectives.

Over the past four months, they have meticulously analyzed the codebases of SN6 and SN9 while striving to lead the respective leaderboards. Although their efforts were met with numerous challenges, they experienced significant successes along the way. They even expanded the validator codebases to include their own logging for detailed analysis, exploring questions like why certain models performed better or why uploading multiple copies made a difference. This groundwork has led them to take the SN9 validator, incorporate their improvements, and publish it. For the time being, SN29 will be a refined version of SN9, with plans for immediate enhancements.

Training and sharing cutting-edge models and code to drive real-world applications:

Coldint is dedicated to making a meaningful impact on the world. Their miners are primarily rewarded for their contributions through the creation and public sharing of trained models. Additionally, Coldint offers incentives for those who share insights by contributing to the canonical miner codebase, whether it’s by identifying bugs, adding features and optimizations, or even rewriting significant portions of the code.

Encouraging both individual and collective contributions:

Coldint rewards miners for incremental improvements, motivating them to consistently share their enhanced work. This collaborative approach allows others to build on existing contributions, accelerating the overall progress. Contributions that enable the combination of individual training efforts into superior collective models are particularly valued, as they unlock the full potential of the community.

Maintaining a steady pace of high-quality iterations:

Coldint operates on a continuous improvement cycle in three key areas:

  • Miners are consistently enhancing models.
  • Training objectives are periodically updated at a consistent pace.
  • Validator and miner codebases are continuously refined.
  • All stakeholders are encouraged to participate in these processes, fostering a dynamic community that thrives on the exchange of ideas, insights, and knowledge.

Ensuring transparency and fairness for all stakeholders:

Coldint is committed to transparency by making miner code, training objective schedules, awarded bounties, and research results publicly available. The community is invited to propose innovative training goals, and measures are in place to prevent exploitation of known “metagaming” tactics. If new vulnerabilities emerge that could misalign miner and subnet objectives, bug bounties are designed to make reporting these issues as rewarding as exploiting them. These bounties are publicized to encourage further bug reports and fixes.

Subnet 29 was created in response to the frustration miners experienced with other subnets. They are highly motivated to examine the validator and weighting logic, which can sometimes overshadow the core focus on training within the subnet. This approach is effective as long as the subnet’s goals align with the validation logic. However, if misalignment occurs, miners may diverge from pursuing the subnet’s intended objectives.

Over the past four months, they have meticulously analyzed the codebases of SN6 and SN9 while striving to lead the respective leaderboards. Although their efforts were met with numerous challenges, they experienced significant successes along the way. They even expanded the validator codebases to include their own logging for detailed analysis, exploring questions like why certain models performed better or why uploading multiple copies made a difference. This groundwork has led them to take the SN9 validator, incorporate their improvements, and publish it. For the time being, SN29 will be a refined version of SN9, with plans for immediate enhancements.

Training and sharing cutting-edge models and code to drive real-world applications:

Coldint is dedicated to making a meaningful impact on the world. Their miners are primarily rewarded for their contributions through the creation and public sharing of trained models. Additionally, Coldint offers incentives for those who share insights by contributing to the canonical miner codebase, whether it’s by identifying bugs, adding features and optimizations, or even rewriting significant portions of the code.

Encouraging both individual and collective contributions:

Coldint rewards miners for incremental improvements, motivating them to consistently share their enhanced work. This collaborative approach allows others to build on existing contributions, accelerating the overall progress. Contributions that enable the combination of individual training efforts into superior collective models are particularly valued, as they unlock the full potential of the community.

Maintaining a steady pace of high-quality iterations:

Coldint operates on a continuous improvement cycle in three key areas:

  • Miners are consistently enhancing models.
  • Training objectives are periodically updated at a consistent pace.
  • Validator and miner codebases are continuously refined.
  • All stakeholders are encouraged to participate in these processes, fostering a dynamic community that thrives on the exchange of ideas, insights, and knowledge.

Ensuring transparency and fairness for all stakeholders:

Coldint is committed to transparency by making miner code, training objective schedules, awarded bounties, and research results publicly available. The community is invited to propose innovative training goals, and measures are in place to prevent exploitation of known “metagaming” tactics. If new vulnerabilities emerge that could misalign miner and subnet objectives, bug bounties are designed to make reporting these issues as rewarding as exploiting them. These bounties are publicized to encourage further bug reports and fixes.

WHO

Team Info

The team closely monitors the Discord channel that is dedicated to subnet 29. Questions, issues, bug reports and suggestions should be posted there, or by DM to the team.

The team closely monitors the Discord channel that is dedicated to subnet 29. Questions, issues, bug reports and suggestions should be posted there, or by DM to the team.

FUTURE

Roadmap

Subnet Creation Changelog:

  • #3379782: Registered 5HHHHHzgLnYRvnKkHd45cRUDMHXTSwx7MjUzxBrKbY4JfZWn for SN29
  • Registered coldint.io
  • Forked pretraining from Macrocosmos
  • Created GitHub repository at Coldint
  • Established WandB project at Coldint
  • Set up HuggingFace profile at Coldint
  • Applied for a Discord channel
  • Cleaned up validator codebase
  • Implemented scoring mechanism in validator
  • Modified sample packing in model evaluation logic
  • Introduced bug bounty, also known as the “Hall of Fame” reward mechanism
  • Deployed two validators and one miner
  • #3413001: Published SN29 metadata on-chain, released GitHub and website

TODO: Immediate Post-Launch Steps

  • Notify actors not using on-chain identity data about SN29 renaming (suggest they adopt on-chain identity data)
  • Implement competitions for targeted training goals
  • Clean up mining codebase and publish on GitHub
  • Explore options for on-chain announcements of validator startups with version information
  • Finalize testing with an arbitrary tokenizer (saving up to 800M of 6.9B parameters on the reference model)
  • Launch first additional competition (weight fraction 0.1) with the arbitrary tokenizer
  • Collect results and feedback on subnet and competition performance

2024 Q3

  • Consult with the community and plan a list of competitions well in advance
  • Research model merging tactics to enhance distributed training potential
  • Draft a shortlist of finetuning targets for niche models
  • Launch the first additional competition for niche models
  • Publish model_surgeon.py, a command-line tool for model modifications

2024 Q4

  • Aim to have 5 niche models in training
  • Provide boilerplate code for web applications and host apps showcasing top models
  • Research external benchmarks to evaluate subnet efficacy

2025 and Beyond

  • Explore Pretrain-as-a-Service and Finetune-as-a-Service commercial opportunities

 

Subnet Creation Changelog:

  • #3379782: Registered 5HHHHHzgLnYRvnKkHd45cRUDMHXTSwx7MjUzxBrKbY4JfZWn for SN29
  • Registered coldint.io
  • Forked pretraining from Macrocosmos
  • Created GitHub repository at Coldint
  • Established WandB project at Coldint
  • Set up HuggingFace profile at Coldint
  • Applied for a Discord channel
  • Cleaned up validator codebase
  • Implemented scoring mechanism in validator
  • Modified sample packing in model evaluation logic
  • Introduced bug bounty, also known as the “Hall of Fame” reward mechanism
  • Deployed two validators and one miner
  • #3413001: Published SN29 metadata on-chain, released GitHub and website

TODO: Immediate Post-Launch Steps

  • Notify actors not using on-chain identity data about SN29 renaming (suggest they adopt on-chain identity data)
  • Implement competitions for targeted training goals
  • Clean up mining codebase and publish on GitHub
  • Explore options for on-chain announcements of validator startups with version information
  • Finalize testing with an arbitrary tokenizer (saving up to 800M of 6.9B parameters on the reference model)
  • Launch first additional competition (weight fraction 0.1) with the arbitrary tokenizer
  • Collect results and feedback on subnet and competition performance

2024 Q3

  • Consult with the community and plan a list of competitions well in advance
  • Research model merging tactics to enhance distributed training potential
  • Draft a shortlist of finetuning targets for niche models
  • Launch the first additional competition for niche models
  • Publish model_surgeon.py, a command-line tool for model modifications

2024 Q4

  • Aim to have 5 niche models in training
  • Provide boilerplate code for web applications and host apps showcasing top models
  • Research external benchmarks to evaluate subnet efficacy

2025 and Beyond

  • Explore Pretrain-as-a-Service and Finetune-as-a-Service commercial opportunities

 

NEWS

Announcements

MORE INFO

Useful Links