A/B testing

How it works

• You create a test in shipLua with a key (for example Tutorial), 2 to 5 variants and a split such as 50/50 or 90/10.
• Each player is assigned once and keeps the same variant in every server and session. Two players in the same server can see different versions.
• Your server script asks for the variant and decides what to show. Anything server code controls can be tested: which UI to clone into PlayerGui, tutorial steps, prices shown, reward sizes.
• If the test is off, the player is not enrolled, or shipLua cannot be reached, the player gets the default you pass. The game never waits for shipLua.

local function startTutorial(player)
	-- "Control" is also what players get if the test is off or shipLua is unreachable.
	local variant = ShipLua:GetVariant(player, "Tutorial", "Control")
	if variant == "Short" then
		ShortTutorial.Run(player) -- a different ScreenGui, steps or script
	else
		ClassicTutorial.Run(player)
	end
end

Reading results

Pick a primary metric (day 1 or day 7 retention, sessions, playtime, payer conversion, Robux per player or error rate) and guardrails that must not get worse. Results show each variant with a 95% range, the lift against control, and a check that the split really came out as planned. shipLua only calls a variant better or worse when the test says so; otherwise it tells you how many more players it needs.

Steps

1. Open the experience, then Experiments > Create experiment.
2. Choose the audience (everyone or a segment), variants, split and metrics.
3. Add the code above and publish the place.
4. Start the test. Exposures count from the first time GetVariant runs for a player.

To change a value for everyone, use a remote config instead.