
Microsoft has announced MAI Cyber 2 Flash for its security products, with eligible Codename MDASH customers scheduled to begin receiving an automatic replacement of the existing MAI-profile model in mid-December. The November 17 announcement describes a forthcoming product rollout, not a generally available standalone model endpoint today.
The official release reports a one-million-token context window and stronger performance-per-cost results on selected security evaluations. The methodology notes are central to interpreting those claims, particularly Microsoft's decision to remove some benchmark tasks because their sources overlapped with training material.
What was removed from the comparison
Microsoft says 92 of CyberGym-E2E's 905 tasks use the same open-source projects and vulnerability reports as some training data. It says the benchmark reference solutions were not used in training. To be conservative, it excluded those tasks for every evaluated model and reported results on the remaining 813.
On that reported evaluation, Microsoft gives MAI Cyber 2 Flash 92.3 percent task success versus 91.5 percent for GPT-6 Astra, with the MAI run using half the reasoning budget. We have not reproduced the comparison. The exclusion is a disclosed methodological precaution, not independent proof that every possible form of overlap has been eliminated.
Our analysis: the denominator belongs with the score
A benchmark result is easier to interpret when readers know which tasks were counted and why others were excluded. Changing the task set may be justified, but the resulting percentage needs to remain attached to that specific population.
The disclosure is useful precisely because it lets others ask narrower questions. How does performance vary across the remaining projects? Which task types remain difficult? Does a successful benchmark attempt produce an actionable finding in a real team's workflow? A headline average cannot answer those questions alone.
Cost needs the same treatment. Microsoft describes its less-than-two-percent cost comparison as an offline evaluation calculation using input, cached-input and output tokens at stated list prices. It explicitly distinguishes that from customer spending. A team's bill also depends on how the product invokes the model and how much work people must review afterward.
Access is through products, with a future start
The rollout is described as exclusive to Microsoft security products. We did not verify downloadable public weights or an open-source license. This story is indexed in Model Watch under the exact name MAI Cyber 2 Flash and publisher Microsoft AI, without inventing a Hugging Face checkpoint.
We also checked the announcement artwork: one promotional social image says less than one percent, while the accompanying release states less than two percent. We have used the documented text and an original methodology diagram rather than repeating the inconsistent image claim. No security scans, exploit tests or customer deployment were performed by us.
Original primary-source reporting and labeled analysis. No model inference, installation, API submission, hardware test or independent benchmark performed.






