A/B Testing MCP Servers (2026): Which Ones Let Your Agent Create, Run and Call a Test

Which A/B testing tools ship an MCP server in 2026, and how far each one lets Claude, Claude Code or Cursor go: create, launch, read results, stop and ship. Checked against each vendor's own docs.

A/B Testing MCP Servers (2026): Which Ones Let Your Agent Create, Run and Call a Test

The short answer: Several A/B testing tools ship an MCP server, but they stop at different points. PostHog, GrowthBook, Statsig and LaunchDarkly are built for product teams: the agent works with experiments and feature flags in your app, and PostHog's docs say the app needs its SDK to serve variants. Optimizely and Convert let an agent read and manage experiments, and Convert's hosted connector creates tests as drafts. Mida creates draft tests and reads results. Humblytics is a hosted MCP server for marketing-site tests: the agent gets test ideas from your page data, creates a no-code or redirect test, reads per-variant results and stops it, with revenue as an available goal. In every case a person should approve the launch and decide on the winner. Checked against vendor docs on 5 November 2026.

Searching "MCP server for A/B testing" today mostly returns Claude skills and articles about testing MCP servers. Neither answers the real question: which testing tool can your agent operate, and how far can it go?

This guide answers that. It sorts each server by stage in a test's life, not by feature list.

What an agent can do in a test, stage by stage

A test has six stages. An MCP server can cover some or all of them.

  1. Recommend. Suggest what to test, from real data.
  2. Create. Set up the test and its variants.
  3. Launch. Start sending visitors to the variants.
  4. Read results. Report how each variant is doing.
  5. Stop. End the test.
  6. Ship. Roll out the winner.

The important question is where the server hands control back to you. A server that creates drafts leaves the launch to you. A server that launches on create puts the approval step before creation. A server with a ship tool can change what all your visitors see.

The servers compared

Every cell below comes from the vendor's own MCP documentation, read on 5 November 2026. "Not stated" means the docs page did not say, not that the feature is missing.

Tool Where it runs Create Live on create? Read results Stop or end Ship winner
Humblytics Hosted (mcp.humblytics.com/v1), or local No-code or redirect test Yes, live at once Per-variant sessions, conversions, conversion rate Yes, stop_split_test No tool. A person decides
PostHog Hosted (mcp.posthog.com/mcp) Experiment plus its feature flag No, separate experiment-launch Yes, experiment-results-get Yes, experiment-end Yes, experiment-ship-variant
GrowthBook Hosted on GrowthBook Cloud (OAuth), or self-run Experiments and feature flags, through API read and write tools Not stated Yes Not stated Not stated
Statsig Not stated on overview page experiment_create Not stated experiment_read Not stated Not stated
LaunchDarkly Hosted (local in federal and EU environments) Feature flags; experimentation tools Not stated Not stated Not stated Not stated
Optimizely Hosted Experiments, flags, audiences Not stated Yes Not stated Not stated
Convert Hosted (mcp.ai.app.convert.com/mcp, OAuth), or local A/B and split URL tests No, drafts Yes Can pause Not stated
Mida Hosted (mcp.mida.so/mcp) A/B tests No, drafts Yes Not stated Not stated
VWO Local (npx) Feature flags and rules Not stated Get Metrics Toggle a flag Not stated

A few details matter more than the table shows.

PostHog covers the whole lifecycle. Its MCP has separate tools to create, launch, read, end and ship an experiment. Each experiment comes with a backing feature flag, and PostHog's docs say your app needs a PostHog SDK to serve the variants.

GrowthBook keeps its server small. It has four tools: two that load skills, and two that read from and write to the GrowthBook API. Its docs list designing and launching an A/B test and reading results as example prompts.

Statsig asks for a confirmation on update tools and on destructive operations. Its docs tell you to review each change before your agent supplies one.

Optimizely works for Web and Feature Experimentation. For Web Experimentation, its docs say variation code changes (custom HTML, CSS and JavaScript) are not supported through MCP. You make those in the Visual Editor.

Convert has two forms. The hosted connector creates "safe draft" A/B and split URL tests, can edit variation code and pause tests. The local server has three access levels: reporting (the default), readOnly, and all, which can create, edit and delete experiments.

VWO's documented MCP server is for its Feature Experimentation product. Its tools manage feature flags and rules. We found no VWO docs for an MCP server that edits web tests.

Humblytics creates a test live. That is the main difference from Convert and Mida. The server's own tool description says the test "goes live immediately and visitors are assigned right away", and tells the agent to show you the recommendation first.

Feature-flag tests vs marketing-page tests

Pick by what you are testing, not by which MCP server looks biggest.

Testing product code: pricing logic, onboarding steps, a new checkout flow. Use a feature-flag tool: PostHog, GrowthBook, Statsig, LaunchDarkly, Optimizely Feature Experimentation or VWO. The variant lives in your code behind a flag. The agent can write that code and set up the experiment in the same session. You need an engineer, or an agent working in your repo.

Testing a marketing page: a headline, a hero image, a CTA, a whole landing page. Use a page-level tool: Humblytics, Convert, Mida or Optimizely Web. The variant is a change applied to the live page, or a redirect to a second page. No release needed.

Humblytics supports two test types:

  • No-code. Each variant is a list of changes: a CSS selector, an operation (text, css, html, image, delete or duplicate) and a new value.
  • Redirect. Each variant is a different URL. See split URL testing.

Goals include click-through, form submission, bounce rate, session time, destination page, external destination and revenue. A revenue goal scores each variant on the sales it brought in, not only on clicks. If you build in Webflow or Framer, these guides show the same tests from the dashboard: Webflow and Framer.

What it looks like: one test in Claude Code

Here is one test from idea to result with the Humblytics MCP server. The tool names are the real ones.

1. Connect.

claude mcp add --transport http humblytics https://mcp.humblytics.com/v1

One connection covers every site on your account. If you have more than one site, the agent calls list_properties to find the right one.

2. Ask for ideas. "What should I test on /pricing?"

The agent calls get_split_test_recommendations with page: "/pricing". Each recommendation names an element, a change type, a new value and a confidence score. The tool tells the agent to prefer ideas scored above 70. Before choosing, the agent can also look at traffic, clicks and funnels for that page with the read tools.

3. Review the plan. The agent shows you the page, the change, the goal and how long to run it. This is your approval point. Nothing is live yet.

4. Create. After you say yes, the agent calls create_split_test with a name, the page, type: "nocode", a goal and the variants. A Control variant is created for you. Visitors start being assigned straight away. You can set auto_stop_days so the test ends itself.

5. Check in. "How is the pricing test doing?" The agent calls get_split_test, which returns sessions, conversions and conversion rate for each variant. list_split_tests shows every active or complete test.

6. Adjust or stop. update_split_test can rename the test or change auto_stop_days. Nothing else. stop_split_test ends it, with an optional reason. A stopped test cannot be restarted.

7. Ship. You decide. There is no tool for promoting a winner. If the variant won, you make the change on your site, or ask your agent to do it in your site's code or builder.

The ab-test-generator skill, served as an MCP prompt by the same server, walks the agent through this flow with live data.

Guardrails: when your agent should not call a winner

An agent that can create tests needs rules. These are the ones we would set.

  • Keep the client's approval prompt on. Cursor asks before running an MCP tool by default. The Humblytics server does not enforce approval itself. The prompt in your client is the safety step.
  • Approve before create, not after. On servers that launch on create, the review must come first. On draft-first servers, review before launch.
  • Check traffic first. A page with few visits can take weeks to give a clear result. Ask the agent how long the test will need at current traffic before you start. See A/B testing with low traffic.
  • Do not stop at the first good day. Early results swing. Let the test run its planned length.
  • Score on the goal that pays. A variant can win on clicks and lose on sales. Where your tool supports it, use revenue as the goal.
  • A person ships. Shipping changes what every visitor sees. Keep that decision with a person, even on servers that have a ship tool.

What the Humblytics agent can and cannot change

The Humblytics MCP server has 45 tools. 35 only read. 10 change something:

  • create_split_test
  • update_split_test
  • stop_split_test
  • save_funnel
  • delete_saved_funnel
  • create_property and delete_property (add or remove a site)
  • get_agents_checkout_link, buy_data_point_top_up and set_auto_top_up (billing; a top-up charges the saved card)

That is the full write surface. The ad account tools are read-only, so the agent cannot edit your campaigns or budgets. On the hosted server, your client sends its own key on each request, and the server stores nothing.

For a wider view of AI testing tools, see AI-powered A/B testing tools. For every analytics and CRO tool with an MCP server, see MCP analytics. Plans are on the pricing page. To connect, start at mcp.humblytics.com.

FAQ

Which A/B testing tools have an MCP server?

As of 5 November 2026, PostHog, GrowthBook, Statsig, LaunchDarkly, Optimizely, Convert, Mida, VWO and Humblytics all document an official MCP server. They differ in what the agent can do: some create drafts, some launch tests, and only some can end a test or ship a variant.

Can Claude create an A/B test?

Yes, if Claude is connected to a testing tool's MCP server. With Humblytics, Claude Code or Cursor can call create_split_test to set up a no-code or redirect test on a page. The test goes live as soon as it is created, so the agent should show you the plan first.

Can an AI agent pick the winner of an A/B test?

Some servers let it. PostHog documents an experiment-ship-variant tool. Humblytics has no tool for promoting a winner: the agent reads results and can stop a test, and a person decides what ships.

Does a test created through MCP go live right away?

It depends on the tool. Convert's hosted connector and Mida create draft tests. PostHog has a separate launch tool. Humblytics create_split_test activates the test immediately and assigns visitors right away.

Do I need a developer to run A/B tests through MCP?

For feature-flag tools, usually yes: PostHog experiments need the PostHog SDK in your app so it can serve variants. Marketing-page tools such as Humblytics change the page through their script, with no code in your app.

Is it safe to let an agent run A/B tests?

It is safe when a person approves each change. MCP clients such as Cursor ask before running a tool unless you turn that off. Keep that prompt on for any tool that creates, launches or stops a test.

Sources

All read on 5 November 2026.