after three hours of playing the game I've realized you've essentially created a slot machine for Bayesian statisticians

Wait, he’s using plushies to “pay” us to play his game, this game is helping gather data about how you sometimes have to trade cost and accuracy in data sets… Well played, well played

As a side note, I've found myself biasing against certain blobs more. For a while, it felt like Sunglasses (both Star sunglasses and normal ones) blobs were cheating more than other blobs, so I was biasing against them.

Kinda neat, I wonder (statistically) how sure one could be with 15-16 flips per blob

His existential dread over sad findings of human behavior might be over.

damn those guys at the top of the leaderboard sure are good, and also committed! ironic that a game called "catch the cheaters!" has no anti-cheat lol. well done anyways - it is a really nice concept to see you covering

May I recommend, if you decide to update it, to add a hard mode where cheaters can have a coin loaded either to heads or tails? I wonder how that'd change things

I love this so much. Human perception of random systems is fundamentally flawed, and I thoroughly expect this "game" to produce a dataset that bears this out. Fischer Yates with a 64-bit Mersenne Twister is the gold standard of pseudorandom number generation for a reason, and is simply as close to truly random as random can get without going quantum with it. 99.9% of the time when someone claims something isn't "random", what they really mean is they "FEEL" it isn't random, that it isn't consistently inconsistent. I don't recall who, but a very smart person once said "if something feels random, it probably isn't." Truly random systems have streaks, and aren't prone to even distributions, especially in such small datasets. Love this. Love, love, love this.

This reminds me of a math teacher [a real one] where he proved to his students the variety in outcomes would be more than they thought. He gave them a homework assignment of doing & tallying 200 coinflips. He told them them didn't have to bother with the coin [i.e. they could "cheat"], but he would know who did do the assignment properly & who didn't. He guessed right right every time, mainly because the "cheaters" would never include enough 7, 8, & 9 consecutive results that would always come up when you 200 coinflips.

On some Android devices, the app may not remember your personal top scores if you close and open the app. If you see this, please comment here or on the play store listing to let me know what device you're on to help me track down the issue. If you do see this, your scores and drawing entries are still stored in the database, though.

"Cheater coins have a 75% chance of landing heads" This is important info.

After over 7 months, the legend came back!

I was literally watching your videos last night and thinking "Man, he hasn't uploaded in a while, huh" Thanks for the video, Primer!

this thing nerd-sniped me to hell and back

This is a great demonstration of Exploitation vs Exploration. This is a common design-of-experiments problem where you want to make a decision with a minimum number of test data. For instance, if you only project 2500 people will click on your advertisement, but you use the first 2000 people to test whichof your 2 ads is the most effective, that only leaves 500 expected people to apply your data results to (ie showing them the best of the two). You essentially lose value on the 1000 people you showed the less effective ad to. In this game, every flip (collection of data) has a cost or weight attached to it. You want to minimize your flips spent while maintaining an average number of flips gained that is greater than or equal to your flip cost. Such a strategy may have a tolerance that allows for a certain number of incorrect classifications depending on the cost of incorrect classification and the gains incurred by a correct guess.

Oh man, I love your blob animations and getting to see them do their thing at the press of a button will be very gratifying :)

Holy crap I was just revisiting this channel. What great timing!

Just got recommended the evolution videos a few weeks back. Working thru them. Immediately was hooked and then bummed that it looked like the channel went quiet. Happy to see you’re still making videos!

It’s always great when you upload!

Excited to see what the full video is about! Multi-armed bandit perhaps?

Now I'm thinking... What happens if you try to cheat in a game about cheaters? Depending on how the game is built it could be trivial (solution or score is calculated/stored client-side) or almost impossible (a server handles everything, you just submit answers). Let's hope it's the second to make it fair ;)

I've been looking at your channel every odd week in exasperation. Glad to see another video!

this is such good timing, came out just after i finished studying hypothesis testing/ type 1 and 2 errors in stats :)

One video per year but still amazing work

I love the hat blobs, you are an excellent content creator. I hope you keep making content, keep doing it at your own pace

Love primer videos. Not only do they have crazy interesting concepts, those blobs have brought more happiness into my life than literally anything else.

Your videos are so interesting to watch! Please make some more :D

The return of the king

I went off the deep end with this game. It's deceptively simple. I did a a full brute force statistical analysis of the optimal strategy. The expected value per round is exactly -0.128856. This means, on average, per blob, if you play perfectly, you'll lose 0.13 flips. This is great because it's about the "house odds" of most casino games. It also means if you play long enough, even perfectly, the game will eventually end. What's really crazy about it is how precise these numbers are. If you change the reward to 16 or the penalty to 29, the expected value is positive (allowing infinite games) If you change the bias of the cheater's coin from 0.75 to 0.7527 the expected value is positive. Games can last forever. 0.0027 is all it takes. All these numbers, 0.5, .75, 15, 30. They're so round you'd assume they'd been picked arbitrarily. But if they were anything else the game wouldn't work. It's on the knife's edge of being infinite or being unplayably harsh. There's something beautiful about that. Well worth spending half my easter revising statistics and optimising algorithms. To work this out I wrote a recursive algorithm that simulated the expected value of every "position" where there's fewer that 400 head or tails. It assumes you play optimally all the way to the 400 flips point then works backwards all the way to the start. (If you work forward you need to evaluate 2^400 games, so back-propagating is the only method that is computationally feasible). Once you work out the optimal strategy you can just store the moves and simulate games. Here's the results of 1M games. n = 1000000 median = 77 mean = 698.4 stddev = 3404.5 max = 220228 With a stddev ~5x the mean, it's an extremely skewed distribution. Most of the games end before 77 points but some outliers go out to the 10s of thousands. Wild. I really like this game. When I first saw it I thought it was just a toy to explain conditional probability (and it is) but there's so much complexity hidden underneath such simple mechanics.

HE’S BACK, AND DECAPITATED AS WELL Edit: Just realized he wasn’t decapitated, his shirt just blends so well into the background.

I just subbed, and theres a new video, such nice timing!

discovered your channel during hte first wave of the pandemic and wait for every single one of your videos since then and i'm watching them in the hour after they come out

Probably better if which side the cheater tries to cheat with is random. Aside from that, really cool concept

Flipping great game! But…are you going to check the leaderboard for cheats using statistics? :)

I'm gonna play it just because the blobbies look so darn cute - great animation!

So glad to see you upload again!

I just accidentally realized that you can really abuse the fact that this is a game if you want highscores, by just guessing the first few rounds without flipping coins at all and then continuing normally once you get a lucky streak... I just wanted to lose quickly and clicked cheater a few times, and miraculously got them all right ^^'

i feel like the game would be more interesting (and realistic) if the cheaters could cheat into either direction

I really enjoy this channel because he explains complex ideas simply so it makes me feel smart

This game is brilliant! For the leaderboard, I would love it if you would add a histogram distribution of all scores with an arrow or line showing where you are, like how leaderboards work in mini motorways or the zacktronic games. That way you can get a sense of your percentile at a glance. It would also be cool to have different modes that score the 4 outcomes differently. If the consequence for cheating is extreme, the cost of injustice for a false positive would be high, so you'd need a strategy that only accused cheaters if the evidence is beyond a reasonable doubt, even if you let some moderate cheaters slip through the cracks. On the other hand, the objective could be quality control, and it is important that only honest blobs make it through, even if some honest blobs get mislabeled as cheaters, like medical trials or quantum physics experiments.

Can you please add a way to speed up the labeling animation? And a way to automate labeling? I know that if I get x H and y T then I'll always label z, for many different pairs of x and y values. At this point it's probably faster to reimplement the game to validate the strategy locally, set up screen capture, ocr and automated clicking to run my algorithm than to manually click through. I'm at 235 score and 448 flips left, thinking of abandoning the run, while the top leaderboard score is over 900. Fun mini game at start but clicking through gets tedious quickly.

Thanks for making this game!

IT'S BEEN A WHILE I'VE MISSED YOUR CONTENT

It's frustrating when they get 5 heads in a row, and it turns out they're not a cheater. The chances of that happening with a fair coin are 3.1%.

No way! You haven't upload for 7 months, but the day after I find you and subscribe you uploads again! Where you just waiting on me this whole time? Lol But seriously, glad to see you upload another video! I fully expect there to be big waits between uploads, but I'm excited to see more whenever you have them ready!

Having corrected homework for a statistics course that taught the concept of confidence limits in probability quite alright. Well timed video.

I fell in love in this game. Simple, satisfying and helps to gather data for nice cause.

SOLUTION(?) (1) You can get the probabilities using the Bayes' theorem. After some rearrangement, I have p(fair|h,t)=4^(h+t)/(4^(h+t)+3^h * 2^(h+t)) where h is the number of heads and t is the number of tails. (2) You have to decide how close to one has to be p(fair|h,t) to call "fair". I tried to find the value analytically, but for me it is too difficult. So I will program a simulation and try different values. (3) My small program tells me that the optimal value is close to call fair blob when p(fair|h,t)>0.75 and call "cheat blob" when p(cheat|h,t)>0.75. Then I get averages close to 850 blobs!!!

Probably not but I’ll still watch anyways

I have been waiting for another video for a long time!

The blobs are amazing!! A great way to gain statistics while making it fun!!! So many of the simulations I’d love to play! In my first game of this, my strategy was purely based on flipping zero times, once or twice, and then using my guessing powers to guess which one it was. It worked effectively, leading me to the 150s and 10 score, but then I had a major losing streak. SO MUCH FUN!

The best strategy by far is to just flip once and if it's heads guess cheater if it's tails guess fair I got 122 second try lol.

He rarely updates videos, but when he does oh are they good.

This channel is a real inspiration

So glad to see you back!

Ok, first strategy, use 10 coins and based on the results, try to get 15 flips. If you are still not sure, it's better to spend another 5 and lose less.

I was wrong!

At last another upload! Thank you 🙏🏼.

Can already tell this will be great

The app became unplayably laggy when the number of blobs became too high, please fix this.

Looking at the leaderboard I'd guess the leaders are scripting or have an insane amount of time on their hands.

This is really fascinating and I am excited by this and the next topic!!

Thank you for making this amazing game. It made me do a lot of thinking and essentially write a bot (which I have never done before) to play the game and select "fair" or "cheater" based on certain outcomes. It was a great journey. I started with simulating probabilities, and then understanding that they can actually be calculated mathematically. Then, I went for the highest accuracy, without realising that value-wise it would be a poor decision. And then after focusing on value, my best "formula" to success was to make a choice every time when the value of a current choice is higher than the mean of the possible next two choices (for example: if current situation is 0 heads & 1 tails, the value is -26, and the mean value of the two possible future situations, which are 1 heads & 1 tails and 0 heads & 2 tails is -27, which means that after having flipped tails only once, it's "best" to select "fair" immediately. I did not expect this when I first coded it). I know little of Bayesian satistics and math overall, so I may either be writing nonsense or child's thoughts for some of you, but I enjoyed the journey, and I am grateful to Primer for making this inspiring me to answer the challenge :) The highest score I got after 6 attempts is ~400

I played this game and I have a prediction: when that the blob is playing fair sometimes the outcomes come out balanced, but often enough there are more tails than heads. What if sometimes, when the game tells us the blob is fair, the coin is biased towards the tails, but Primer is not revealing it to teach us something about psychology once he shows us the data.

If this was a secret ad for those plushies, it worked because I just bought one. Keep up the great work!

Always a good day when I see a new Primer video

If you spam click the cheater / fair button while you're losing, you can accidentally "vote" on an extra, unshown blob. This results in impossible remaining flips such as -40, and also doesn't present the reset button or an option to enter the high score. It does not matter whether animations are disabled or not.

Looking forward to the next vid after a long time

i would love to see more of these games that are based on the other simulations.

Good too see u back😀

Can't wait to see the new video :)! Really nice topic.

I can't believe I am so late! I know you don't post often but I absolutely love your videos

Bro your videos are amazingggg I have like zero brain cells but I promise that watching these have doubled that 💛

Love the concept cool video Justin :)

What a cool analogy, I look forward to the video!

this is awesome, i didn't think he would upload again to be honest

my boi back with some fire vids

thank you for doing this

Wow, powerful and insightful!

I stumbled across this while I was looking for something to practice the new Excel Lambda function with. Long story short, I've written a solver in Excel. In my simulated results, it can get scores over 100 before my tables runs out of entries. It assumes a 50% chance of a cheater and a 20% chance of the coin being "dropped". (Are these numbers listed anywhere?) My methodology was pretty simple. You want to increase the expected value of a correct answer, but only while the increase is worth the cost of the flip. It's a stateless game, so all you really need to consider is how many heads and tails you've seen in this round. So I started by determining the odds of getting a given combination of heads (H) and tails (T) for both a fair coin and a biased coin. This is done using a binomial distribution. (built in to excel) From their you can determine the chance that the blob is cheating. This is determined using Baye's Theorem, which in this case is just a fancy word for a weighted average. Now you know the chance of a cheater for any given distribution. You need a 66% certainty either way to break even. But what we really want to do is maximise certainty while minimizing the flips. The real question is whether to flip once more, or take a guess. The easiest way to approach this is to consider the expected outcomes of each choice. If you take a guess now, you will win or lose based on the known expected value. If you flip once more, you'll lose 1 flip and then either 1) lose the coin or 2) get a heads or 3) get a tails. If you get a heads or tails you'll know more about the round but might actually reduce your certainty in your position. You should consider the expected value of the resulting positions (more on that in a bit). You could also assume a 50/50 chance of heads or tails when calculating your expected value, but you actually know more than that. There's a chance the blob is a cheater which will increase the odds of getting a heads. So we can now make an informed decision about whether we're better off guessing now or trying our odds with a flip and guessing then. But that still leaves out a big piece of the puzzle. What if we keep flipping, can our odds improve and how does that inform our immediate decision? The standard tool in the programmer's toolbox for this kind of problem is recursion. Now, I'm sure a mathematician could produce some infinite series and prove it converges to a simple function, but I'm not a mathematician so I'm going to make the computer do the thinking. It turns out, if you recursively evaluated the expected value at each state, you're encouraged to flip more than if you naively take the expected value of a guess at each state. As you add more levels to the recursion, it does have diminishing returns (it converges) but not as quickly as you might expect, so it's probably worth optimising. What's also surprising, from my findings, there's also no definitive "end" to a flipping sequence. If the distribution hovers around the midpoint of the two expected distributions, it stays in your best interest to keep flipping. This is counterintuitive and I'm not sure I've gotten that entirely right (again, not mathematician). Still, whether I've done the mathematics right or not, my little excel script seems capable or almost limitless play. I'll have to take a crack at the leader board. This was a great thought experiment and even though it's dealing with entry level probability material it did genuinely challenge me to think creatively and the results surprised me. It also gave me a great excuse to abuse Excel and use its functional programming in ways it was never intended. Good stuff.

OH MY GOSH HES BACK!!! I almost crapped myself when I saw this in my notifications.

just remembered this channel existed and remembered at a perfect time

Can't wait for the next video!

statistical hypothesis testing with the binomial distribution looks like it’ll come in great here !!!

I love the blob accessories

It has been a loooong time :) Nice to See you :)

You're alive. Amazing.

Can't wait to see your next video!

Keep up the good work man love the videos

I love your videos so much! Your amaaaazing!

First William Osman and now Primer. What a great day!

been a while, nice to see a vid again.

Wow, I've always loved your channel, and only now I learned that you use Unity to create your videos! I would love to see a backstage video on how you do it!

After 7 months you made a video FREAKING UPLOAD DUDE

finally these videos are too good

Yay! A new primer video!

You came back at the right time.

Since you're coding in unity for most of these projects I would love to see you package some of the stuff you've made into a broader "game". Your videos feel like the foundations for the type of game everyone wanted spore to be and it could be a great ancillary revenue source for you, maybe allow the download as a Patreon tier? Big project for sure, but your videos and work is so high quality I'd love for you to get the money you deserve out of it.

Magnificent Justin!

The legend came back again