Math is under attack by stupid people

The number 1 stupid person is Donald Trump. He micromanaged the budget of the National Science Foundation to realign it with his favorite topics: AI, computer science, and quantum computing, even offering a $200 million dollar prize for a working quantum computer. The Republican hate of DEI has caused drastic cuts in grants and brought about reductions in support for graduate students. Xenophobia has prevented a flow of talent from abroad. A friend of mine said this effort should be called Make China Great Again.

Now openAI has shown that it has no clue about how to conduct mathematical research by dropping the solution of 372 problems, the intellectual equivalent of Taylor Swift releasing 30 albums at once. I spent some time reading the article in Scientific American about the greatest hits (and you can too if you give them your email address). It would have made much more sense to concentrate on one or maybe two dozen achievements and write them up in a way that people can read and understand the achievements. If this was done then there would have been a sudden burst of progress. As it is we will have to sift through the wheat to find the chaff, and as openAI admits probably 10% of the solutions are simply wrong.

A mathematicians who work at Anthropic called it the greatest moment in the history of mathematics. Well maybe so, but I am pretty sure he has the sign wrong. Never has there been such a steaming heap of crap dumped into the literature that will set back mathematics for years. We all knew AI could kill us all but we didn’t expect how or when.

Math is more than solving problems

At this point we do not know which of 372 new theorems in openAI’s upload are breakthroughs or which ones were solved by putting together insights created by human minds and uploaded to the internet, where they were found by AI bots and assembled like legos. But I know that (i) These agents are the not the Ramanujans of the 21 century. (ii) It will take dozens of humans hundreds of hours to decode each new morsel of knowledge.

None of this really matters. Mathematicians are a community. In addition to working together to decipher the mysteries of mathematics, we teach undergraduate students to understand well-established methods for solving problems, guide them in navigating ther undergraduate careers  and initiate graduate students into the arts of research, teaching and mentoring.

It is possible that 20,000 AI agents could have collaborated to solve the mysteries of Navier and Stokes without spying on other people’s work, but they could not have done it without the recent published breakthough of Córdoba and Martínez-Zoroa,. That holds true for the recent progress on the problem of showing no percolation at the critical value in any dimension. There would be no solution without the conjectures of Gady Kozma and Shahaf Nitzan that set-up the framework for solving the problem. See Gil Kalai’s blog

Yes, the landscape of mathematical has suddenly changed, but humans are needed now more than ever. Robots may be able to run faster than Usain Bolt but they can’t figure out how to stop before crashing into the wall at the end of the course. For a laugh use google to find a video. Much of the progess AI has made in math was simply come  from mining the gold tand the less precious minerals hat has accumulated over the last 100 years in the mathematical literature. Large language models are talented aggregators of information, which offer mathematicians an unprecedented ability to retrieve that information, compared to crawling the graph of papers liked by citations. But if they really understood what they read then they would be able to write their proofs n a way that we can understand them.

What is herd immunity?

The purpose of this note is to explain some simple math that is very relevant to current events.

Every disease has a basic reproduction number, R. It is the average number of secondary infections created by an infectious individual (in an otherwise susceptible population). Early in covid R was 2.5 but it grew to 5 or more as the disease evolves. Measles has an R that is much higher. According to Google’s AI overview “it is widely cited as being between 12 and 18.”

The other number we need is u, the fraction of the population that is unvaccinated. If Ru < 1 herd immunity has been achieved and the disease cannot spread. If we suppose R = 14.28 then this means 93% of the population has to be successfully vaccinated. The word successful refers to the fact that sometimes vaccinations do not produce the desired immunity. My wife Susan was vaccinated against whooping cough but caught it from her Christian Scientist neighbors.

As usual reality is more complicated than math. Even if Ru < 1 there can be groups of people who for religious, political, or other reasons do not believe in vaccination. This creates areas in which the disease can spread.  The 2025 Texas measles outbreak was centered in a close-knit and under-vaccinated Mennonite community in West Texas’ Gaines County, where low vaccination rates were a key factor in the rapid spread of the highly contagious virus. The outbreak started within this community but eventually spread beyond it to other Texas counties and into New Mexico. 

Florida has recently made a dangerous decision to no longer require children be vaccinated for measles and other diseases in order to attend school. While it will take several years until herd immunity is lost, the fact that school populations are stratified by age it will almost immediately create subpopulations in which diseases can spread. The problem will grow worse over time. When it gets bad it will be very hard to raise vaccination levels in the population to stop it. .

Vaccine policy needs to be informed by science. It is not enough for RFKJr to say “I am not taking away anyone’s vaccines.” It is important to have people in HHS and CDC who advocate for vaccination, and educate the public about the need for them and their safety and effectiveness. Last but not least we need to remove barriers to getting vaccinations such as the current need for a prescription to get a covid vaccination. Fortunately that virus has evolved to be less deadly but it can still cause serious disease in people with “high risk health conditions.” These people can get vaccinated but they will be much less safe if there is a significant amount of  covid infection  in the population as a whole.

Griffin vs. Riggs: what are the odds the result will change if 65,000 votes are thrown out?

Post updated March 6 and 7 — see the  end.

Justice Allison Riggs maintains a 734-vote lead over Republican rival Jefferson Griffin after more than 5.5 million ballots were cast and two recounts. Griffin is seeking to have 65,000 ballots thrown out. Today I will answer a question no one has asked: What is the probability he will win the election if the votes are thrown out? The answer is 2/1000.

A reversal may sound extremely likely, but the square root of 65,000 is 255. Suppose we have 5.5 million ping pong balls in a swimming pool. Exactly half have R for Riggs and half have G for Griffin. We then draw 65,000 ping pong balls out. What is the probability that the outcome of the election is changed? If we count balls with R as +1 and balls with G as -1 then the election result is change if the sum of the numbers which we call S is > 734, since this is the net number of votes lost by Riggs.

In the jargon of probability these balls are drawn without replacement, but given the large number of balls, this differs very little from drawing with replacement, which is just flipping 65000 coins and counting +1 for heads and -1 tails. On the average the net change in the outcome is 0, or in the terminology of probability the expected value of S = 0. Of course the resulting sum is unlikely to be exactly 0. In elementary probability class we learn how to compute the size of the typical deviation of S from 0, which is the standard deviation. In the case of 65000 independent random variables that are +1 with probability ½ and -1 with probability ½ the standard deviation is the square root of 65,000 or 255.

The central limit theorem allows us to estimate the probability the number of votes lost by Riggs, S > 734. This value is 734/255 = 2.878 standard deviations above the mean. Using my calculator function normalcdf I can calculate using the normal approximation that the probability the outcome of the election does not change is 0.997998, or the chances that the election outcome will change is less than 2 out of 1000 or 0.2%.

Taking this reasoning further. Let us suppose that throwing out the 65,000 votes results in a loss of 1530 votes for Riggs. This result is 6 standard deviations above the mean. Using the calculator again gives a probability of 0.00000001 that this occurs. Thus if throwing out the votes results in a change of more than 1530 votes in the outcome then we are absolutely certain that the votes chosen to be challenged were not chosen at random. They are a biased sample of ineligible voters that contains more Democrats than one would. This is hardly surprising since the Republican party made the list, but this observation provides one more reason that the votes should not be thrown out.

One reader questioned the assumption of indpendence of party and registration status. One practical consideration is that if we don’t do that we can’t compute anything. In ecological terms, independence is a null model. Once we reject it then we ask why. One explanation is that democrats are more incompetent at filling out the registration form, but it seems more likely that the Repubicans who made the list were more likely to put Demicrats on it.

————————————-

As I have thought about the situation more I think that is a convincing argument for not throwing out these votes can be based on statistical data. If the list consisted of say 3000 voters all of whom were Democrats, it would be ridiculous to throw out these votes which would hand to election to Griffin.

I don’t have access to the list of 65,000 names and their parties but if for instance it consisted of 30,000 Democrats, 20,000 Independents and 15,000 Republicans this would show a bias since the fraction of registered voters of the three types are 32% D, 38% I, 30% R. Information quoted in Riggs’ emails already points to an unexpectedly large number of college students on the list.

. I suspect that at the end of the process when the case reaches the NC Supreme Court it will be decided on party lines and Griffin will be elected by the same group that decided partisan gerrymandering is legal. As the actions of the legislature in the years that they have had a veto proof majority indicate, Republicans are not swayed by arguments about what is unfair. But I guess the large fraction of them who believe that Trump won the 2020 election indicates they don’t believe in statistics either.

 

Presidential Polls are Flawed

This was written Sunday October 27, 2024 and will not be revised.

  1. The president is not elected by popular vote.

I remember watching Headline News on CNN a few days before the 2012 election and hearing that the election between Obama and Romney was a statistical dead heat. Each candidate was predicted to have 47% of the vote. However, the results in the Electoral College were not close Obama 332, Romney 206. People who forecasted based on a state by state basis (Nate Silver and a talented amateur at Princeton) concluded on the days before the election that the probability Obama would win was 99%.

  1. Even when the difference between the candidates is less than the margin of error of the poll it still contains information.

Let’s start with the question: what is the margin of error? It is often reported, but rarely with the sample size of the poll which is needed to be able to check the calculation. Some examples with both pieces of data are Franklin-Marshall for Penn 794/4.3; Emerson College for North Carolina 1226/3.6, and for Wisconsin 800/3.4; Marist for Arizona and Georgia 1193/3.9.

If Ɵ is the estimated frequency of votes for Harris then using standard formulas from an intro states class the radius of the 95% confidence interval the margin of error is 1.96(Ɵ(1-Ɵ)/n)0.5.  Since Ɵ is usually in (0.4,0.6) then 0.24 < Ɵ(1-Ɵ) < 0.25 and replacing 1.96 by 2 we can simplify this to 1/n0.5 and our data points become

n,error               800,  0.0353       1200, 0.288

somewhat smaller than the reported margins of error.

  1. Frequencies should be reported to one decimal place and should be recalculated to remove responses that are not one of the two major candidates to make it easier to compare predcitions.

As some readers have noticed I have not gotten to #2 yet. That is coming soon. Bloomberg, the one poll that I have seen which reports frequencies with one decimal place had on October 23:

Harris/Trump    H/(H+T)           P(H win)   538 ave.

Michigan            49.6/46.9           51.4                     79.2         H 0.6

Penn.                  50/48.2               50.9                     67.7         H 0.3

Nevada               48.8/48.3           50.3                     59.2         H 0.2

Wisconsin          48.3/48               50.2                     51.6         H 0.1

Arizona               49.1/46               50.2                     51.6         T 1.8

  1. Carolina 48.5/48.8 49.4                     43.9         T 1.2

Georgia               48.4/49.9           49.3                     35.2         T 1.5

If the uncertainties look large, it is because they are based on one poll. Once one has the average of several polls then the uncertainties decrease. Of course comparing Bloomberg with 538 average shows that there are sampling biases. The Washington Post is even more discordant. They have Harris leading in all states except Arizona. I got these figures from an article in Forbes but when I went to check them the Post wanted $2 to read the article.

Combining these projections to calculate is difficult without a computer and a good programmer. 538 uses simulations to conclude that, when this was written: Trump wills with probability 0.54 and Harris with probability 0.45.

https://projects.fivethirtyeight.com/2024-election-forecast/

One final selling point for people who publish polls, if you make predictions like this you can’t be wrong.

Trump 323, Biden 215

This blog post is based on the previous one The Arithmetic of Presidential Elections. There we used data from the last three presidential elections to argue that there are Red States with 170 electoral vote, Blue States with 215, and Swing States listed below with 153.

Turning to https://projects.fivethirtyeight.com/polls/president-general/2024/

for data, we see that Biden trails in every swing state

 

States               Votes                 Biden                 Trump

Ohio                   18                        36.8                    46.1

Georgia             16                        38.3                    44.8

Arizona              11                        38.9                    43.9

Iowa                   6                          32                        50.4

Carolina           15                       38.0                    45.2

Florida               29                        36.5                    46.1

*Penn.               20                       41.6                    43.6

*Wisc.                10                        41.8                    42.8

Nevada              6                          38.7                    42.6

*Michigan        16                        41.2                    43.0

*New Hamp.   4                          42                        44

 

Since this predicts Biden will only carry the Blue States this gives the prediction above.

Flipping the four closest states marked with * would result in 273 – 265 but likely the polls do not yet fully reflect his abysmal debate performance.

 

The Arithmetic of Presidential Elections

In November of 2012, I remember watching Robin Meade on Headline News saying that the race between Obama and Romney was a statistical dead heat. Both were projected to get 47% of the popular vote. Meanwhile a guy named Nate Silver on his web site 538.com was predicting based on state-by-state data that Obama would win by a comfortable margin and he did.

2012    Obama            332      Romney          206

2016    H. Clinton       227      Trump             304      Other   7

2020    Biden               306      Trump             232

My question here: is what will happen in 2024?

By now it is widely recognized that there are Red States, Blue States and Swing States. To identify these we gathered data by looking at the Wikipedia articles on the 2012, 2016, and 2020 Presidential Elections, where it can be downloaded into a spreadsheet. Rather than use our “knowledge of politics” to classify the states, we declared a state RED if it had gone Republican in all three elections, BLUE if it had gone Republican in all three elections, and SWING otherwise.

Cleaning the Data

Dealing with real data is annoying. One immediate headache is that in Maine and Nebraska two electoral votes are decided by the voters of the state as a whole, and the others are determined by votes in the various congressional districts. This is handled differently in the three data sets. Since ME-1, ME-2, NE-1, NE-2, and NE-3 were consistently used for the districts, we made the brilliant decision to use ME-0 and NE-0 for the statewide electoral votes.

The New and North states were also a problem. In one data set they are alphabetized by their abbreviations: N.C., N.D., N.H., N.J., N.M., and N.Y. while in the other two the names are written out. Before I realized this, there were some very surprising patterns in the voting of New Yorkers. Finally when I thought I had found all of the differences I saw that in the 2020 Maryland was abbreviated Md. so it swapped places with Massachusetts, and in all three data sets West Virginia was written as W.Va and hence came before Washington (the state).

Our methodology

To create an ordering of the redness of states, we looked at the percentages of votes for Democrats minus the percentages of votes for Republicans, and ranked them by the sum of the numbers for the three elections. This produced the following classification:

Red States (170 Electoral Votes)

(Reddest first) NE-3, Wyo., W.Va., Oklahoma, Idaho, North Dakota, Utah, Kentucky, Arkansas, Alabama, South Dakota, Tenn., NE-0, Kansas, Louisiana, NE-1, Montana, Miss., Indiana, Missouri, Alaska, South Carolina, Texas

Blue States (215)

Minnesota, Colorado, ME-0, New Mexico, Oregon, New Jersey, Delaware, Washington, Illinois, Virginia, Conneticut, ME-1, Rhode Island, Vermont, New York, California, Mass., Maryland, Hawaii, D.C. (Bluest last) On the average 85% of people in DC voted Democratic in the last three years, but that may change this year

 

Swing states (153)

(almost red) Ohio, Georgia, Arizona, Iowa, ME-2, North Carolina, Florida, NE-2, Pennsylvania, Wisconsin, Nevada, Michigan, New Hampshire (almost blue)

Reality Check

According to Wikipedia “areas considered battlegrounds in the 2020 election were Arizona, Florida, Georgia, Iowa, Maine’s 2nd congressional district, Michigan, Minnesota, Nebraska’s 2nd congressional district, Nevada, New Hampshire, North Carolina, Ohio, Pennsylvania, Texas and Wisconsin.”

 

Moneyline Wagering II

Given that sports betting has come to North Carolina I am going to revisit the topic and derive some general formulas. The take home message is that while you might think you could make money betting against people who know less than you do. It is just like going to a casino.  The typical moneyline wager looks like

Fav(orit)e  -B      (Under)Dog +A

What this means is that you have to bet $B on Fave to win $100, while if you bet $100 on Dog you win $A. Let p be the probability Dog wins.

For the bet on Fave to be fair we need 100(1-p) – Bp = 0 or p = 100/(100+B)

For the bet on Dog to be fair we need -100(1-p) + Ap= 0 or p = 100/(100+A)

In practice A< B so if 100/(100+B) < p < 100/(100+A) both bets are unfavorable.

For a numerical example, in the sweet 16 of the 2024 NCAA tournament we saw the bet

Marquette -300, NC State + 230, so both are unfavorable if

0.25 = 100/400 < p < 100/330 = 0.303

In this particular case NC State won, which is not incredibly unexpected since when you roll two dice then the probability of a total of 9 or more has probability 10/36 = 0.2777

Another way of looking at this is through the money bet. If a fraction x of people bet on Fave then

When Fave wins the average winnings are 100x – 100(1-x) which is < 0 if x < 1/2

When Dog wins the average winnings are -B x + A(1-x) which is < 0 if x > A/(A+B)

In our concrete example both bets are bad for players if 0.434 = 230/530 < x < 1/2

While the sports book has no control over the probability that Dog wins, they can control the fraction of money bet on the favorite by adjusting the odds over time, or using their knowledge of previous bets to choose good values of A and B.

When A/(A+B) < x < ½ then the sports book has an arbitrage opportunity. They will make money under either outcome. In the theory of option pricing it is assumed that arbitrage opportunities do not exist (or if they do are short lived), and based on this one can derive what is the right price for a “derivative security” such as a call or put option based on the stock prices.

If you want to learn about this look at Chapter 6 of my book Essentials of Stochastic Processes which you can download in pdf form from my web page. https://services.math.duke.edu/~rtd/EOSP/eosp.html

My goal here is to make two points (i) the sports books are not gambling: they have things set up so that they win money no matter what the outcome is, (ii) even though you are gambling against a group of people and it seems that you can win money if you are smarter than they are, that is an illusion. Like casino gambling, things are set up so that all of the bets are unfavorable. So like the TV commercials say look at sports betting as a way of enhancing the fan experience not as a way to make money. However with betting available 24-7, in game parlays and bonus bets designed to get you used to betting a lot of money, this like the lottery, is a scheme designed to take money from people who are gambling with money they can’t afford to lose.

Remembering Krishna Athreya

 

As I write this, he passed away about a week ago. While the end of a person’s life is a sad time, it also provides an opportunity to reflect on the past. The first memory that came to my mind was of sitting in Sid Resnick’s office in the Stanford Statistics department in 1974 (or 1975.) A group of us got together once a week to make our way through the new book on Branching Processes by Athreya and Peter Ney.  It was a welcome modernization of Harris’ 1963 book that started the subject. There were crisp proofs of the basic facts about Galton-Watson processes, Markov branching processes (with exponential lifetimes), the age-dependent case (general lifetimes), and multi-type branching processes. Over my career I have loaned this book to my graduate students many times to help them learn the subject. Remarkably I still have it but it is a little worse for wear.

#2 on my list of Athreya’s greatest hits is A new approach to the limit theory of recurrent Markov chains which appeared in the Transactions of the American Math Society, a paper that was written with Peter Ney in 1978. Again this is a contribution to an area founded by Ted Harris. While Markov chains on discrete state space are well understood, on a general state space numerous pathologies arise. Harris’ genius was to identify a class of these chains that have a tractable theory and cover a number of examples.

There is an elegant analytical theory described in the book by Revuz. However in 1978 several researchers, including Esa Nummelin who later developed a book based on this approach, had the same idea at the same time. I remember attending a session of talks at the 1978 meeting on Stochastic Processes and their Applications and hearing three talks on the topic. This was devasting for a Ph.D. student in the audience who was working on this for his thesis.

The idea is simple but brilliant: a Harris chain can be modified to have one state that is hit with positive probability starting from any state and having one such state is enough to carry out all the usual theory for the discrete case. Given this hint I am sure you can work out the details for yourself. I was so excited by the idea that I put it in the Markov chains chapter of my graduate text book.

Returning to a more traditional narrative: Krishna Athreya received his Ph.D. in 1967 from Stanford where he worked with Sam Karlin, a legendary probabilist with an impressive pedigree: son of Bochner, who was a grandson of Hilbert, and the mentor for 44 students including Tom Liggett and Charles Stone among many others. Athreya’s thesis topic was Multitype Continuous Time Markov Branching Processes and Some Classical Urn Schemes. Soon after he got his degree Athreya and Karin worked on Branching processes in random environments. Two papers were published in Annals of Mathematical Statistics in 1971 since the Annals of Probability which began in 1973 did not yet exist.

These two papers like many in Athreya’s top 20 most cited on MathSciNet contain a number of ideas that have not been fully explored. An example is the work with his Ph.D. student Jack Dai on random logistic maps. Last but not least, I would like to mention his 1994 paper on large deviations for branching processes which contains material that working probabilists should know. Athreya has left an impressive mathematical legacy that will enrich your life and research if you have the time to read it. It is sad that there will be no more work coming from him, but I hope others who read this will be inspired to continue his work. .

The Tea-Cup Problem

Here’s a little problem to test your skills at combinatorial probability.

You have a set of six cups and saucers. Two are NC State red R, two are UNC light blue b, and two are Duke dark blue B. You place the saucers in a line on the table RRbbBB. Then a blind man comes in and puts the saucers on the cups in random order. Let M be the number of cups that match the color of the saucer they are on. Your job is to compute the distribution of M.

To get you started I will specify a probability space which is the first step in solving any problem of this type. I once thought it was good to number the cups but a student in my class this year taught me it was better to treat the two cups of a given color as indistinguishable so we have 6!/(2!2!2!) = 90 outcomes instead of 720. To help check the solution note that not only should the probabilities sum to 1, but we must have EM = 6(1/3)=2. In the next paragraph I will start to reveal the solution starting at 6 and working down, so if you want to discover it on your on you should stop scrolling.

P(M=6)=1/90. In our probability space there is only one outcome where the cups all match, which is better than the situation when the cups are numbered and there are 2 x 2 x 2.

P(M=5)=0. If say the 2R match and 2b match then the 2B must match so 5 is impossible

P(M=4)=12/90. Matching 2-2-0 is impossible by the reasoning for 5, so we must have 2-1-1. There are 3 ways to pick the color with two matches, and for each color with only one match 2 choices of where the matching cup is. The rest of the outcome is now forced, e.g., RRbBBb.

P(M=3)=16/90. Matching 2-1-0 is impossible so we must have 1-1-1. We can pick the locations of the matching cups in 2 x 2 x 2 ways. The other three nonmatching cups must be either BRb or bBR

P(M=2)=27/90. We can have 2-0-0. Once we pick the double matching color in 3 ways the rest is forced, e.g. RRBBbb. We can have 1-1-0. We can pick the color with no match in 3 ways and 2 x 2 ways for the location of the matching cups. Suppose dark blue has no match. Then the two B cups must be on R and b, but there are 2 ways to put R and b on the B sauces for a total of 24 + 3 = 27.

P(M=1) =24/90. We can pick the location of the matching cup in 6 ways. Suppose it is the first R saucer. The second Red saucer can be B or b (2 ways). If it is B then we have bb on the Blue saucers, and we can have RB or BR on the b sauces (x2). If it is B then we have BB on the blue saucers and we have two possibilities on the B saucers, but autocorrect in Word will not let me type them.

P(M=0) =10/90. We can have BB on the Red saucers and then must have RR on b and bb on B. The situation is similar for bb on Red. This gives 2 outcomes. If we have {Bb} on the red saucers then we must have {Rb} on b and {Rb} on B where the set braces indicate we have not specified the order, so there are 2 x 2 x 2 = 8 outcomes.

1+12+16+27+24+10=90, 6 x 1+12 x 4+3 x16 +2 x 27+1 x 24 = 180 (so the mean is 2).