Good information

Hey @dljones555 - Thanks so much for the thoughtful feedback! Lots of great ideas here, and we’re excited to see you're already thinking along the same lines we are!

• Prompt names/identifiers: Heck yes! You can already add a name: field at the top of each .prompt.yml file to identify it. It’s used throughout the UI.

• Loading prompts in Microsoft.Extensions.AI: We're actively working on integrating with major OSS packages now! Awesome to hear this one specifically is of interest - I'll run it by the team. Any others that are specifically on your mind where you'd love an integration?

• Non-developer collaboration: 100% aligned. That’s a core reason we have you write the prompts in YML, so that they can be extracted in our UI and iterated on by non-developers there. If there's folks you work with who have ideas to make this even more intuitive, definitely send them our way for feedback!

• Evals in CI: Yes! You can run evaluations in GitHub Actions today! Read more here, or check out a sample action here!

• Graphing eval performance: Love this. It's something we're exploring but no plans to break ground immediately on it.

Keep the feedback coming! This is super helpful as we continue to improve the experience!

Kate,

Thanks for replying. I've explored the prompt management ecosystem even a little further.

• I would hope you would support MCP and their schemas and API's for Prompts and Roots. If I have it right, GitHub Models could then be an https:// root and you would support their /list and /get endpoints.
• It seems there is already fragmentation in prompt formats but MCP will be the standard.
• I should be able to author a prompt or in GitHub Models and specify its root in a frontier model chatbot, other AI tool or agent code and list and get available prompts, specify the securely maintained system prompt by name as instructions and get context based on message history of user/assistant pairs and then continue with the chat session and responses.
• Have you thought about supporting LangSmith Hub? I have also seen Amazon Bedrock and their Converse API. PromptLayer seems to have traction and an API as well. Microsoft has its Prompt Flow? offering in Azure AI / Foundry.
• Some seem to even allow pulling and pushing a system prompt and execution. That should never go to the publicly visible client but on a secured backend via an API seems ok. Some of these are not allowed prompt content to the client or only user and assistant role prompts.
• Based on created evals, can you generate code for the evals that one can running locally using say Microsoft.Extensions.AI.Evaluation libraries?
• More input: have you considered prompt signing?

To me, it seems there are prompt creation and eval tools geared towards engineers, but not workflows that are friendly to allowing non-engineer to work on prompts. There are also more consumer friendly prompt management tools that don't integrate into prompt as code approach or CI/CD workflows. A blend of both approaches might be the sweet spot.

Are you positioning GitHub Models and the prompt tools as friendly for non-engineers to help with the prompt development lifecycle?

Prompts are becoming key knowledge assets for organizations and other roles like QA, business and operations people should be involved in authoring and testing these and they should seamlessly be reusable in agent code by name.

I took an attempt a file based PromptLoader (peer coded with GitHub Copilot mostly) that I want to develop further to integration with MCP SDK's for support the roots and prompts specs. https://github.com/dljones555/promptloader

David L. Jones
Azure AI Engineer Associate
Sponsor my projects on GitHub

Hi David, really appreciate you sharing such a thoughtful and well-researched response! You've clearly dug in here!

While we're still in the early stages of exploring where GitHub Models can go, your points about MCP, other prompt formats, and others are incredibly helpful context as we think about where we go next.

I also totally agree on non-engineers in the prompt development lifecycle. That’s something we’ve been hearing more broadly from large enterprises too.

I’ve shared your post internally with the team, we're actively shaping thoughts on MCP now so the timing is great. Really appreciate you calling those out.

Thanks again for contributing to the discussion here - please keep it coming!

Hey @krishnanmk, great question!

To clarify: Once you enable billing, all usage (whether via the API or the in-repo playground/compare tabs) is immediately billed according to your pricing plan. There's no longer a free tier that gets used up first once you enable billing. This is because paid users get higher rate limits right away (e.g., more requests per minute), so you're getting added value from the start.

That said, usage in the public playground at github.com/marketplace/models is always free! It’s not tied to any org, so it’s not billed, but the tradeoff is that rate limits are much lower.

Totally fair point about messaging. We just added a banner to the public playground to help clarify this, and we’re happy to keep improving the experience.

Hope that helps! Feel free to reach out with any other questions!

Hey @Nassergmr can you tell us more about what you're trying to do here? With a few more specifics I can be more helpful!