Meadows relates a familiar experience: identifying a problem and a solution, but being powerless to enact change. Not that she made no change; just not as much as she hoped. She was a technocrat in spite of herself, an environmentalist with a computer model, a metrician who understood The Tyranny of Metrics.

Meadows wrote The Limits to Growth, which said that you can't make infinite tin cans from finite tin. People keep saying this, as in More Everything Forever, as in Survival of the Richest. Surely it's right, as far as it goes. In what must be a huge coincidence, estimates for when society will collapse are close to estimates for The Singularity.

There are interesting bits in the book about how oscillations result from delays in information and responses, overcorrecting needing to be corrected.

I enjoyed how she had no patience for “the invisible hand of the market” somehow producing optimal outcomes as a result of greed. I don't think I'd heard the term she used, “the invisible foot,” to refer to the cause of the negative outcomes that come. (Bounded rationality was also mentioned.)

A version of Meadows' leverage points is included. The point of modeling systems might be to change them, after all. It reminded me somewhat of Nudge. The “Nudge Unit” idea has faded considerably these days, I think. Meadows was more forthright in offering that modeling systems hadn't let her change them as much as she had hoped to.

Her number one leverage point feels philosophical. It is about “transcending paradigms.” I was amazed. I had already agreed with her largely on philosophy of models (“Everything we think we know about the world is a model,” etc.) But here she was offering what I think is very nearly my personal philosophy on relativistic foundations. The most confusing bit is that she offers it as the highest-leverage point. She must mean changing the foundations that others adopt? Maybe? Otherwise it sounds like the “leverage” here is the first part of the Serenity Prayer: “accept the things I cannot change.”

Mindsets and Models

• Everything we think we know about the world is a model.
• Our models do have a strong congruence with the world.
• Our models fall far short of representing the real world fully.

Guidelines for Living in a World of Systems

1. Get the beat of the system. (This means learn about it, with data, etc. - Aaron)
2. Expose your mental models to the light of day.
3. Honor, respect, and distribute information.
4. Use language with care and enrich it with systems concepts.
5. Pay attention to what is important, not just what is quantifiable.
6. Make feedback policies for feedback systems.
7. Go for the good of the whole.
8. Listen to the wisdom of the system.
9. Locate responsibility within the system.
10. Stay humble—stay a learner.
11. Celebrate complexity.
12. Expand time horizons.
13. Defy the disciplines.
14. Expand the boundary of caring.
15. Don't erode the goal of goodness.

I identify with one of his major points, which he frames as arithmetic being about comparison. I have usually thought of this like “numbers as data visualizations.” So we have to work out 8 times 6 in base 10 (rather than base 8, say) in order to compare it to other values in base 10. Fractions with common denominators, units your audience is familiar with, etc.

Lockhart says you shouldn't drill math facts, or force people to use standard algorithms, and certainly you shouldn't write the one you're carrying at the top of the next column. I have some hesitation here.

For one thing, I think of a well-worked calculation as a kind of proof: using the known single-digit facts and properties of the base-ten system, you demonstrate that the product really is what you say it is (for example). It's hard to say this would be lost, since it typically isn't taught this way in the first place, but I'd like for it to be found.

Second, I think if we don't ensure people are competent calculators themselves, and we also don't teach the effective use of some electronic calculator, there's a gap. When I was a kid, using the standard algorithm to divide was like reaching for a calculator: I set it up, and then could basically do the calculation without expending real mental energy, then move on with the result. My fear here is that a student is trying to do something interesting, then gets to a calculation and is basically stuck because they both aren't proficient calculating by hand and don't have a convenient calculator to reach for. Even for me, my use of calculators is so ad hoc: my phone, or a Python REPL, or a Jupyter notebook, all feeling clumsy and not well integrated into a workflow. In my thinking about Linear Algebra, I think a big part of why the teaching of it is so strained is that the calculations are annoying by hand, and there's no great convenient, commonly available calculator for things like SVD.

But certainly I like the emphasis on enjoyable math. Before the current AI scare, math was thrashing about calculators and calculation for years. Lockhart is ahead of the curve, in a sense.

I disagree with Lockhart's philosophy in his “Machines” chapter, I think. (Or is he joking?) He makes an interesting comparison among the mechanical “carry pin” (move the tens digit once for every turn of the ones digit) and the transistor and the neuron. And then he suggests that a tally counter is in a sense conscious. This is some Daniel Dennett daftness, in my opinion.

It's a very nice book though. I think on page 168 is says 12/35 instead of 12/30, but I otherwise didn't notice typos. Very nicely printed and bound. Some question about who the audience is. I guess me?

Arithmetic is the skillful arrangement of numerical information for ease of communication and comparison. (page vii)

Mathematics, in particular, is an explanatory art form, and ultimately all of its structures arise as "informa-tion carriers" for the purpose of explaining a pattern or idea. (page 7)

Nearly throughout, Lockhart uses “leftovers” instead of the common “ones.” I like this; “ones” is a little weird as a plural, and you could say that 20 has 20 ones, after all, but this is not what is usually meant.

I guess my real point here (and with this book in general) is that there are many good strategies for encoding and manipulating numerical information, and you can use them in any way you see fit. (page 73)

Changing ells and fathoms to meters is one thing, ... (page 77)

Huh! Ell like “elbow,” or like “cubit!”

Notice that there is a subtle yet important difference in status between the two numbers here. When one writes 5 x 8, the 8 is the actual quantity we are adding—the price of a single muffin, say. The 5 is a counter; it indicates the number of copies we want. In a sense, the 5 is "operating" on the 8, and not the other way around.

So there is an asymmetry to multiplication: 5 x 8 does not mean the same thing as 8 × 5. Surprisingly, however, the two totals happen to be the same. (page 88)

This is what I mean when I say that arithmetic is the art of rearranging quantities—noticing and taking advantage of specific features of your counting problem to make your life easier and to have a bit of amusement while you're at it. (page 91)

First of all, let's understand that 7 × 8 is not a question and 56 is not an answer. Seven times eight is a number, and it is capable of being represented in a great many ways. At the moment it is held as seven groups of eight, and a user of an octal system would be quite pleased and would not feel the need to "do" anything to it. When we ask, "What is seven times eight?" what we are really asking is, "How can seven groups of eight be rearranged into groups of ten for ease of comparison with other numbers similarly grouped?" Numbers couldn't care less what grouping size you happen to use and neither do mathematicians. Numbers are what they are, and they do what they do; your desire to compare is the issue, and your culturally determined choice of representation system is quite secondary. (page 93)

Arithmetic can be a gateway drug for mathematics. (page 97)

In every practical, real-world scenario (including even particle physics and cosmology), there is always going to be some threshold of accuracy beyond which things become pointless. For one thing, your original data is always approximate anyway. So scientists and engineers, carpenters and seamstresses, bakers and farmers all use arithmetic frequently, but the most important skill is knowing what degree of accuracy is appropriate—in particular, when remainders can safely be discarded. If there is such a thing as an accepted convention, it would be this: don't bother being more accurate than your data. If the situation calls for you to divide 125.26 by 3.8, it would be absurd to work hard to produce a figure like 32.9631579, when the original numbers are only given to one or two places after the decimal point. I would probably just go with 33, seeing as it's so close.

Generally, when someone writes a number like 3.8, what they mean is that all they can guarantee is that it's around 3.8, somewhere between 3.75 and 3.85, say. If more were known (or cared about), then maybe we would have a more accurate estimate like 3.83, which would again only tell us the number up to a certain tolerance—the final digit always being a bit iffy. The decision of how accurate one's measurements should be, and thus how many so-called significant figures to provide, is a very basic one. It really just depends on what you are doing and why.

_You are in the laboratory measuring the amount of liquid contained in three test tubes. The volumes (in milliliters) are found to be: 30.25, 27.12, and 32.62. After combining them, you find the total volume to be just a hair over 90 milliliters. Should you be surprised?__

I keep thinking I should go make the estimation calculator I've thought of making...

This is pretty much what scientists and engineers spend their time doing. They construct simplified mathematical models of their problems and then work on them using the precise and elegant patterns that mathematical objects obey. Then they translate back to reality, making any necessary approximations as they see fit. In this way, mathematics can be viewed as a "model shop"—a source of convenient simplified models of reality.

For mathematicians, however, the correspondence works the other way. The objects of interest are the abstract imaginary patterns, and reality serves merely as a source of modeling material. If I want to think about perfect imaginary rectangles, I can use a desktop or a piece of paper as a crude, admittedly inaccurate physical model. Of course, we understand that such a clumsy prosaic object could never contain any actual mathematical truths (in particular, it cannot have exact measurements), but it can still give me ideas and lead me to observations and understanding nonetheless. So for mathematicians like myself, reality is the model shop. This was more or less Plato's view: real things are merely the shadows of the true, idealized mathematical forms. (page 181)

Fun!

Mathematics is the study of pattern. And of course I mean pattern in the abstract—the patterns of Mathematical Reality. (page 182)

After homework was done, each kid worked a sheet of five third-grade sample problems from the Noetic Learning Math Contest, which I had heard about from a friend. (I also had fourth and fifth-grade papers for them, which they were interested in tackling as well, but we didn't spend much time on them in person on Monday.)

Every kid missed at least one problem one way or another, and it was interesting to hear them share their solution methods and correct themselves based on what they heard from their friends (and occasional adult re-framing).

Then they did a problem as a group, which was the grade four problem from Gordon Hamilton's Unsolved K-12, which I happened to learn of because I have a copy of his book Socks Are Like Pants, Cats Are Like Dogs. It's a fun presentation of the Collatz Conjecture.

I feel a little corny doing a bunch of patter for a problem, but the kids seemed to really enjoy it, especially the “splat.”

It also makes it a lot more engaging that one problem is beatable. Our group quickly found a family of solutions and then was uninterested in finding more for that case. But they spent considerable time looking for counter-examples to the Collatz Conjecture, which was a fun way to practice arithmetic as well.

When I eventually explained that nobody had yet found a counter-example, their reaction was not at all to give up; one kiddo instead suggested trying 100. It seemed that, as intended, having them tackle an open problem was interesting and motivating.

Most math textbooks focus on the formality and have less focus on the spirit, with face-to-face teachers often being the ones to convey the spirit that the text couldn’t, or didn’t. (page 396)

I finished How to Bake Pi wanting more Category Theory; what luck that Cheng had already written the book I wanted! Here's how she illustrates (with typical great clarity) the goal of the book. (The names on the right are authors of more difficult Category Theory books.)

I think she absolutely delivers, and she delivers with spirit and understanding and humanity. It's a math textbook, starting very gently and getting a little difficult by the end, all written in a confident and kind first-person voice.

I like the “Things to Think About” that she intersperses. They're a little like exercises, but they're nearly always followed immediately by explained solutions, in a way that doesn't interrupt the flow of the book. You can stop and work things out yourself first or not. It's a great technique, I think.

Cheng has explanations I haven't seen elsewhere. For example, a monoid is usually defined as a set of things with an associative binary operation and an identity. That doesn't seem very “mon” at all. Cheng explains that a monoid is a simple one-object category, with all its morphisms starting and ending on that anonymous object, so they all compose. So those arrows are “the things in the set” and composition is the “binary operation.” Suddenly it makes sense to call this thing a monoid! And composition is a generalization of binary operations in general. So cool!

I almost wish Cheng went further into monads and so on, but the book is already 400 pages. Maybe I'll try some of the other books she recommends as follow-ons.

mathematics involves the twin disciplines of logic and abstraction. (page 24)

Thus truth is not absolute but is contextual, and so we should always be clear about the context we’re considering. This idea is central to category theory. (page 44)

It turns out that a world in which everything is equally valid is not very interesting, and is more or less the same as a world in which nothing is possible. (page 51)

Technically an Euler diagram. This will be the case every time I refer to a Venn∗ diagram. (page 58)

This is a footnote, and then for most of the book any time she says “Venn diagram” it has that asterisk.

Apparently the distinction is that a Venn diagram always shows every possible overlap, even those that are empty? Who knew?

See, math isn’t just about equations: sometimes it’s about inequations. (page 74)

the main thing to take away now is that composition of arrows is a generalization of the concept of binary operation. (page 99)

This is cool!

Turning the arrows around is called taking the dual in category theory, and we’ll look into that more in Chapter 17. (page 114)

Later we will talk more about the idea of turning all the arrows around in a category. If we turn all the arrows around but leave everything else the same it’s called the dual category. It hasn’t really changed the structure of the category, it’s just presented it differently. The first version and the dual version above encapsulate the same information. This small and apparently irrelevant flip in perspective actually turns out to be rather profound and often useful; we’ll come back to it in Chapter 17. (page 116)

For emphasis, the previous, more rigid version [than partially ordered set] is called a totally ordered set. And these are sometimes abbreviated to “toset” and “poset” because mathematicians are so lazy. (page 121)

We have now seen these three ways of putting a category structure on the natural numbers, and each one gives us a very different shape of category.

1. Using factors.

2. Using size.

3. Using addition.

In the first two cases, the objects of the category were the natural numbers, but they used different arrows. The third case is a bit different as it now takes the arrows to be natural numbers, and there was just one object which was a sort of dummy object. This is a form of “dimension shift” that can feel a little destabilizing but which is very powerful. We’ll come back to it. Categories with only one object are quite special so they deserve a name.

Definition 11.2 A category with only one object is called a monoid. (page 126)

Possibly the most striking consequence of having only one object is that all arrows are composable. We mentioned this in Chapter 8, when we were talking about the fact that composition is a generalization of a binary operation. When there is only one object, composition actually is a binary operation on the arrows. (page 126)

A group is a monoid in which every element has an inverse. (page 130)

For example, equations can have symmetry: if we switch the x and y in the equation \( x^2 + 2xy + y^2 = 0 \) then the equation stays the same (assuming that addition and multiplication are commutative). This is the beginning of the idea behind Galois theory, which connects equations with groups via some “abstract symmetry” involving their solutions. Group theory is a classic research field of pure mathematics with a vast body of knowledge and techniques. For this reason many advances in other areas of math have come from finding a way to make a link with group theory. This is often indicated by putting the word “algebraic” before a field of research, because group theory is a form of algebra. (Algebra in high school means something completely different and rather mundane, usually to do with manipulating equations and solving for x. I’m sorry that the word “algebra” has been taken over in this way in high school.) (page 132)

This has helped me understand why some poor white men are particularly angry about the theory of privilege, as they are told they are very privileged, but they see people with technically less privilege doing better than them in society. So they think the theory of privilege is nonsense, because they don’t feel the manifestation of any privilege. I think it is much more productive to understand the source of this anger rather than simply get angry in return. This poset and the map that doesn’t preserve order are how I found clarity around this issue, and I have found that the abstraction helps to keep emotional reactions out of the way and enable a less divisive discussion. I currently consider this to be the most important “application” of category theory I have found, even though it’s really an application of categorical thinking rather than of a piece of category theory itself. (page 156)

It is worth looking out for any non-order-preserving functions in life. Whenever you have two hierarchies on the same set of people and those hierarchies don’t agree it is a potential cause of antagonism. (page 156)

These morphisms between categories are so important that they have an actual name rather than just “morphisms of categories”: they are called functors. The property of preserving identities and composition is called functoriality. (page 159)

Sometimes I like to say “all equations are lies” but really I mean that all equations involve some things that are in some sense not the same, apart from the equation x = x, which is useless. The point of an equation is to use the sense in which two sides are the same to pivot between the senses in which they aren’t. Category theory says that the “sense in which they’re the same” can be more relaxed than equality as long as it still gives us a way to pivot between two sides. (page 166)

Categories have one extra dimension over sets. That extra dimension is where we express relationships between objects, and we are now going to use that dimension to express this particularly strong form of relationship, that is, categorical sameness. (page 167)

One theme of category theory is that we take definitions that refer to elements of sets, express them using only morphisms in a category, and can then immediately apply them in any category we like, not just Set. (page 176)

Let’s start with monoids. Intuitively two monoids should count as “the same” if there’s a way of perfectly matching up their elements in a way that also makes the structure match up, so that all we’ve really done is re-label the elements. We’ll now see that this is indeed what the formal definitions give us. (page 177)

It's sometimes confusing when she talks about “elements” of monoids; she has to mean the morphisms, in the categorical view, which are elements, but “elements” still tends to make me think of objects. Or maybe she's just flipping easily to the view of monoids as “having” elements that combine via a binary operation other than composition.

Incidentally, isomorphism is sort of the “wrong” notion of sameness for topological space. When it’s defined directly in topology (rather than via categories) it gets the name homeomorphism (not to be confused with homomorphism) but often the type of sameness we’re interested in is the kind we talked about earlier where you continuously deform things, and this is called homotopy equivalence. This doesn’t mean our notion of isomorphism is wrong, it means that the basic category structure we’ve put on topological spaces is not sensitive enough to detect the more nuanced type of sameness. But it’s still a decent starting point. (And homeomorphisms aren’t completely useless.) (page 183)

Thus any time we invoke an equality between objects in a category we have done something not very categorical, or not really in the spirit of category theory. When you get used to categorical thinking I hope you will feel a sort of shudder of distaste any time you see an equality between objects, and at least a slight ringing of alarm bells any time you see an equality at all, just in case. Incidentally this distaste for equalities makes me feel particularly put out by the assumption that mathematics is all about numbers and equations. Not only do I not do equations in my research, but I am actively horrified by them. (page 184)

That process we just went through of replacing an equality with an isomorphism is a typical part of a process known as “categorification”. This usually refers to a process of putting an extra dimension into something to give it more nuance via some morphisms. But we don’t just give it some morphisms—we then take every part of the old definition, find all the equalities between objects, and turn those into isomorphisms instead. Then, just as with categorical uniqueness, we may need the isomorphisms to satisfy some conditions of their own. (page 185)

It’s like the classic research seminar where the speaker tells you that something you’ve never heard of before is just an example of this other thing you’ve never heard of before. (page 186)

Like: A monad is just a monoid in the category of endofunctors.

Surjectivity is like injectivity but the other way round. The formal set-based (non-categorical) definitions don’t look so much like “other way round” versions of each other, but with the categorical version we can see a very precise sense in which they are the other way round from each other: they are duals, which means that in the categorical definitions all we need to do is turn all the morphisms to point the other way. If we do this to the definition of monic we get the definition of epic, which is a categorical version of surjectivity. (page 197)

Sometimes (in worlds without commutativity) things are cancelable on one side but they don’t have both sides of an inverse, and this is what happens with monics and epics. Given f s = f t then f being monic says we can “cancel” it to conclude s = t. If f is epic we can cancel it from the other side, so that if s f = t f we can deduce s = t. (page 200)

I like my matrix diagrams for thinking about this kind of thing...

Here's a 2-by-1 matrix, rank one, and it's intuitive that moving right to left “expands information” in a way that we should be able to undo on the left with a 1-by-2 matrix. But you couldn't do that on the right, because it “squished away” some information from the left.

And note, being injective from right to left is equivalent to being surjective left to right.

Now here's a 3-by-2 matrix, still rank one. Can I reason about this the same way? Going right to left it isn't injective any more, because we're squishing away a dimension of the input. So there's no left inverse. Still a pseudoinverse. And the squish means it can't be surjective left to right, so same deal on the right.

Pretty nice diagrams, I think!

The Axiom of Choice says that it is possible, and this axiom is logically equivalent to saying that all epics split. (page 202)

Axiom of Choice strikes again!

We know that 2 = 1+1 in N, and by the definition of monoid homomorphism we must have f (1 + 1) = f (1) + f (1) so we have no choice about where 2 goes: it has to go to f (1) + f (1). (page 203)

Mac Lane (in)famously wrote “All concepts are Kan extensions”. I would say that all concepts are universal properties—the question is just in what context? Furthermore, all universal properties are initial objects somewhere. So all concepts are initial objects. (page 224)

Initial objects are dual to terminal objects, so you could say it the other way too...

Sometimes the initial objects are to do with freely generated structures, such as with monoids where the initial object, N, was necessarily the monoid freely generated by a single element 1. Freely generated algebraic structures are related to the use of functors to move between categories. A freely generated structure is typically produced by a particular type of functor, a “free functor”, which in turn has a universal property expressed via a relationship with a “forgetful functor”. Those relationships are called adjunctions; we will not get to those in this book but I thought I’d mention the term in case you want to look them up. (page 225)

Category theory puts so much structure in place in advance that often a large part of a construction or proof is just a question of type-checking. (page 229)

The use of “type-checking” is interesting; it reminds me of Curry-Howard isomorphism stuff.

Anyway, this is what we mean by saying that terminal objects are the dual of initial objects. Indeed some people call them final objects and cofinal objects. (page 234)

I think there might be a typo on page 236, where in the first diagram it seems like \( C_1 \) and \( C_0 \) are switched. (But it's possible I'm misunderstanding something here.)

...

Ah! I went and looked it up, and there are errata! I was right! And I missed so many more that others have found! ❤️

The abstract explanation for this approach involves limits in categories of algebras for monads. (page 254)

I strongly believe that after a certain point in this proof it’s much easier to write your own proof than try and decipher the symbols involved. I think many (or most or all) mathematicians read research papers this way—reading just enough to try and work out the proof for themselves. (page 260)

Hmm! I am too lazy a reader, maybe...

When we first introduce addition to small children we might give them a number of objects, and then some more, and see how many there are altogether. So to do 3 + 2 we give them 3 objects and then 2 more objects and then take the disjoint union of those two sets; we just don’t say it like that, typically. (page 265)

Another good candidate for Math Made Complicated...

Multiplication is defined by taking categorical products. This shows that addition and multiplication are dual to each other, which is a curious point of view given that multiplication is also repeated addition. (page 265)

I'm not sure now how much the addition and multiplication interpretations are forced...

A general principle is that universal properties involving cocones (rather than cones) are much harder to construct for “sets with algebraic structure”. [Footnote: In case you’re interested in looking it up: this statement can be made precise in terms of algebras for monads.] (page 266)

There are some unhelpful issues of terminology around universal properties. One is that coproducts of groups are typically called “free products” in group theory, and products are called “direct products” (there is something else called a “semi-direct product”). It took me ages to work out that the free product of groups was actually the coproduct and I wish someone had told me. Some people think it’s pedagogically better if you work these things out for yourself, but I was the kind of student who was convinced that I must be confused and doing something wrong if I thought of something that hadn’t been told to us, so I was afraid to ask, for fear of being thought stupid. One solution to this is to encourage all students to be more arrogant and believe in their own brilliance; another is to explain more things and make sure you never make anyone feel stupid if they ask a question. (pages 267-268)

Yes!

Do we have something like unique factorization into primes? If so what are the “primes”? The last one is quite well studied for groups and is particularly straightforward for Abelian groups, where there is a “fundamental theorem of Abelian groups” rather like the fundamental theorem of arithmetic (which says that every natural number can be uniquely expressed as a product of primes). (page 269)

Okay, an Abelian group is a commutative group...

Wikipedia says this fundamental theorem is that “every finitely generated abelian group G is isomorphic to a direct sum of primary cyclic groups and infinite cyclic groups. A primary cyclic group is one whose order is a power of a prime.” So it is like the fundamental theorem of arithmetic.

The idea is that if we didn’t have the morphisms to C, the canonical thing to do would be to take the product A × B, which is the set of all ordered pairs. However, to make sure the diagram commutes we take the “fibered” product, which means we restrict to those ordered pairs (a, b) such that a and b are mapped to the same place in C. (page 273)

This “fibered” thing is one of those words I feel like I've seen and not understood, in the past.

Believing that something is true before proving it can be dangerous as it can result in leaps of faith rather than rigor, but if the belief comes from deep structural understanding it can guide us towards rigor. (page 284)

Love this.

If you can’t remember what the definition was at this point, I hope you might feel you could start making it up for yourself using the idea of preserving structure. I think this is the best way to get math into your brain: understand the principles so that you can create it for yourself. This is why I think that structural math shouldn’t involve memorization, though some people do a bait-and-switch and claim that by memorization they mean “put into your memory”. I think that’s not what memorization typically means, and that we should keep separate the idea of rote memorization (without meaning) and the sort of process I’m calling structural, where you deeply understand a structural principle so that it becomes part of your consciousness. (page 290)

Love this too, and it's an interesting distinction...

The sorts of categories I’m calling “quintessential” are sometimes called “walking” (as coined by Baez and Dolan), like a man with a mustache that is so big it looks like the man exists only to support the mustache: he’s a walking mustache. (page 296)

The fact that an isomorphism is still an isomorphism after applying a functor is called preservation (page 297)

So we can start with any set A and produce the monoid of (finite) words in the elements of A. This is called the free monoid on A. (page 301)

Structure that is preserved by all functors is generally called absolute. (page 304)

Functors between large categories of mathematical structures are sometimes how entire branches of mathematics get started. (page 306)

Algebraic topology also uses other functors to groups, and to more complicated categories. Homology and cohomology are different ways of having a functor to groups (in fact, to Abelian groups) and to chain complexes, which are a series of Abelian groups, one for each dimension, and some “chain maps”, special group homomorphisms relating the different dimensions. (page 307)

Another example of a branch of mathematics based on a functor between large categories is linear algebra, which importantly relates the category of finite-dimensional vector spaces to the category of matrices. (page 307)

This notion of “sensible” is not rigorous at all, but it’s a feeling about the situation that we can have as humans, and developing those feelings is an important part of developing as a mathematician. (page 311)

At the top we have a category of “data equipped with a particular kind of structure”, together with structure-preserving maps. In good situations the free and forgetful functors are in a special relationship called an adjunction, which can be expressed via universal properties in various ways that are beyond our scope here. The composite UF is then a functor from the category of underlying data to itself, and has many excellent properties. It is a prototype example of what is called a monad. Monads are then an abstract way to study “sets with structure”, and more generally the category of data at the bottom could be something other than Set, in which case monads give us a very general way to study algebraic structure. In fact, at an abstract level this is a definition of algebra: something that can be studied via monads. (page 313)

This is an abstract explanation of the disagreement that happens when some people complain about the prevalence of men sexually harassing women, and some other people retort “Women do it to men too”. Women do do it to men too, but there is a structural difference when it’s a group that holds structural power in society harassing a group that does not hold structural power in society, as opposed to the other way round. Those who refuse to acknowledge this difference are looking at an isomorphism of sets as in the bijection above, and those who acknowledge the difference are looking at the impossibility of the isomorphism of categories based on that bijection, even if most people don’t express it in quite that way. (page 320)

We say that F and G have to be parallel, which means they have the same endpoints. The shape of α is then called globular. (page 331)

Page 333 talks about presheaves, which are related to sheaves, which are another math thing I've seen around and not really understood.

This is often the point of having two equivalent characterizations in abstract math. One is a concrete set of conditions to check to ensure that some abstract principle holds, and the abstract principle is then the thing that enables us to do things. The key is to know how the two are related. This is often generally referred to as coherence conditions. (page 338)

Note that the prefix “pseudo” is often used in category theory when something holds up to isomorphism. Here F and G are like “inverses up to isomorphism”. (page 338)

Cheng recommends: Emily Riehl, Category Theory in Context, Dover, 2017.

Also (page 416) Steve Awodey's Category Theory, which she says is even more accessible.

A category is called indiscrete (or chaotic) if it is equivalent to the terminal category. (page 342)

Indiscrete categories are used in the theory of cliques; I’m saying that in case you want to look them up. (page 342)

Discrete and indiscrete categories have opposite universal properties. They are analogous (ideologically and formally) to discrete and indiscrete spaces in topology: in a discrete space all points are considered to be far apart, and in an indiscrete space all points are considered to be close together. (page 342)

This is why category theorists are happy saying “a monoid is a one-object category” where others might get upset about the level shift. Those who get upset might accuse category theorists of not being rigorous, but I reckon we are being rigorous, we’re just comfortable using “is” to refer to an equivalence of categories, not just an isomorphism. (page 344)

As a friend of mine put it: anyone can complicate things, but it takes real intelligence to simplify them. And I will note that simplifying something is not the same as what I’ll call “simplisticating” them; that is, there is a difference between making something simple and making it simplistic. (page 351)

It looks complicated when you write everything out but I hope to have conveyed a sense that all it really means is that everything you could possibly hope to be true is true—as long as you hope for the right things. This is one of the reasons I love category theory: it’s a place where all my dreams come true. (page 367)

We then found that we had to go up a dimension to 2-categories to deal with natural transformations, and although we haven’t done it, this is also what enables us to express general limits and colimits formally, as well as opening up the way to adjunctions and monads. (page 367)

The pentagon and the triangle are the axioms we need for a weak monoidal category, to ensure that we can manipulate structure isomorphisms as if they were equalities. In fact, as this is a one-object 2-category, those are the axioms we need for a weak 2-category, also called a bicategory. The structure isomorphisms are also called coherence constraints, and the question of how constraints in general interact is one of the big questions of higher-dimensional category theory. It is the question of coherence. (page 383)

The small set of generating axioms can seem a little arbitrary sometimes, and I personally think this is one of the reasons abstract math can seem baffling. By contrast, the global presentation makes sense in some fundamental way. (page 384)

The periodic table of n-categories was proposed by Baez and Dolan as a table showing the sorts of structures that arise with different levels of degeneracy in weak n-categories. (page 388)

Neat!

Higher-dimensional categories are difficult, and get more and more difficult as the dimensions increase. They are already difficult when n = 3 and are not very well understood beyond that. (page 388)

These generalized operations are often expressed by means of monads. We have not really talked about monads but they are special functors which are particularly good for generating algebraic structures “freely”, as we did for the free monoid construction. Once we have generated the structure freely we then say “right, now we want this structure actually to have a value in our underlying data”. So for a monoid we would start with an underlying set A, generate the free monoid on it (all the words) and then say: given any word abc, say, I want this actually to have a value in my original underlying set. For example in the natural numbers we would say “2 × 4 × 3 is a valid possible multiplication, now what does it actually equal?” Abstractly this amounts to giving a function free monoid on A A. (page 392)

I think she's getting at how mu maps from a product space down a layer? So it might map 2 × 4 × 3 to 8 × 3 to 24. But maybe this isn't quite what she's talking about? Hmm.

This is something that we can do using algebras for monads. (page 392)

For me there is one final, even more abstract attraction of infinite-dimensional category theory: I think of it as a fixed-point of abstraction. If science is the theory of life, and mathematics is the theory of science, and category theory is the theory of mathematics, I think that 2-category theory is the theory of category theory, and 3-category theory is the theory of 2-category theory, and so on. Finally, however, higher-dimensional category theory is the theory of higher-dimensional category theory. We have reached a pinnacle of abstraction. Or perhaps the heart of abstraction, or its deepest roots.

Perhaps the pinnacle and the heart and the deepest roots are the same, and that, to me, is the joy of abstraction. (page 395)

Mathematics is founded on rigor and precision. (page 396)

Most math textbooks focus on the formality and have less focus on the spirit, with face-to-face teachers often being the ones to convey the spirit that the text couldn’t, or didn’t. (page 396)

Morphisms are higher-dimensional than objects and give us more nuance. More generally, every higher dimension we involve gives us even more nuance, along with more complications. But the aim is to learn how to deal with the complications in order to benefit from the added nuance. I think this is important in life as well. Arguments in life have become too black-and-white. I wish we could all become more higher-dimensionally nuanced in our arguments in math and in every part of life. (page 400)

Not enough neurons

Modern neural nets are big, but it's hard to compete with the number of possible concepts. If we have to use a separate neuron for everything, we need a huge number. An extreme version of this idea is the unit/value hypothesis as in Feldman and Ballard's 1982 paper.

People have sometimes found single neurons that seem to correspond to a single interpretable concept, as in the 2012 cat neuron or the 2017 sentiment neuron.

But the numbers just don't work out. GPT-3 had a vocabulary of around 50,000 tokens, and that immediately went to a layer of 12,000 nodes. It isn't enough to give every concept a node.

Distributed representations

You can represent many more concepts if you use combinatorial approaches, for example like Hinton's “coarse coding” (1986).

But because these representations are not orthogonal, it can be hard or impossible to disentangle them if you have more than one active at the same time.

Quasi-orthogonal dimensions

There are a lot of ways that things are strange in many dimensions, and one of them is that as you get into lots of dimensions there are exponentially many nearly perpendicular directions.

This is usually presented as coming from the Johnson–Lindenstrauss lemma, but the most directly applicable paper I've seen is the 1993 Quasiorthogonal dimension of Euclidean spaces.

My little experiment: The horizontal axis is the actual number of dimensions. The vertical axis is the number of quasi-orthogonal dimensions (within 10 degrees of perpendicular to all the others) based on drawing random (Gaussian) unit vectors. By 700 real dimensions, there are easily 7,000 nearly orthogonal directions.

This gives us a truly mind-blowing number of axes, and the ability to compose and decompose reasonable numbers of them when activated simultaneously (“superposition”).

The structure of language

If quasi-orthogonal dimensions provide the capacity, a model still needs to be able to relate things and build complex concepts from simpler ones. A version of this is the “binding problem” (1986): what goes with what?

One conceptual solution was “treelets” (2003), which looks just like X-bar theory or the sentence diagrams that elementary schools used to teach.

What we have now is attention, which, together with positional encodings (2017), provides a mechanism for the hierarchical processing these suggest.

And it works

Anthropic has shown that things like this happen in practice, and has used these ideas, for example to successfully extract interpretable features from their models.

This post is based largely on work from Anthropic's Transformer Circuits Thread. I presented a lightning talk on this topic in 2025.

“The Four-Way Test of the things we think, say or do” was written in 1932 by Herbert J. Taylor as part of turning around his aluminum company. It was adopted by Rotary International in the 1940s. I have kept the capitalization that they seem to use.

The Four-Way Test is very reminiscent of an 1872 poem by Mary Ann Pietzker, “Is it True? Is it Necessary? Is it Kind?” Hard to say whether there's a connection.

I think the form of the test, as questions rather than prescriptions, is an interesting alternative to a lot of the lists of virtues, and laws that mostly say what not to do.

1) If a man commits murder, this man will be executed.

2) If a man acts like a bandit, he will be executed.

3) If a man keeps someone captive, this man will go to jail (and) pay fifteen shekels of silver.

...

23) If a man causes the loss of another man's eye, he will pay thirty shekels.

24) If a man breaks off another man's tooth, he will pay thirty shekels.

...

33) If a … beats a gentleman's daughter and causes her to miscarry, he will pay thirty shekels of silver.

34) If she dies, this man will be [executed].

35) If a … beats a gentleman's slave girl and causes her to miscarry, he will pay ten shekels.

36) If the slave girl dies, he will give slave for slave.

...

B2) If a man dies and has no sons, an unmarried daughter [will become] his heir.

...

D3) If a lion mauls a man and a physician heals him, the silver for it will be five shekels.

D4) If a physician extracts a kidney stone, the silver for it will be five shekels.

...

E6) If a slave marries his mistress (owner), he will be executed.

The introductory line is from a translation of P432130. The laws are a selection from Miguel Civil's translation of over 80 items. These date to 2112 BCE – 2094 BCE.

...

195) If a son strike his father, his hands shall be hewn off.

196) If a man put out the eye of another man, his eye shall be put out.

197) If he break another man's bone, his bone shall be broken.

198) If he put out the eye of a freed man, or break the bone of a freed man, he shall pay one gold mina.

199) If he put out the eye of a man's slave, or break the bone of a man's slave, he shall pay one-half of its value.

200) If a man knock out the teeth of his equal, his teeth shall be knocked out.

201) If he knock out the teeth of a freed man, he shall pay one-third of a gold mina.

202) If any one strike the body of a man higher in rank than he, he shall receive sixty blows with an ox-whip in public.

203) If a free-born man strike the body of another free-born man or equal rank, he shall pay one gold mina.

204) If a freed man strike the body of another freed man, he shall pay ten shekels in money.

205) If the slave of a freed man strike the body of a freed man, his ear shall be cut off.

...

This is from the L. W. King translation of the Code of Hammurabi, which was written circa 1753 BCE. The whole code is far too long; these are just a few selections, with just eleven of 282 “laws.” I put a bit from the epilogue at the start, to give a flavor for the very long sections outside the laws themselves.

I'm a little unsure about including “laws” like these in the virtues project. It highlights how petty laws can be, how verbose, how primitive. Did they, in this ancient time, have ideas about virtues? They are called “laws of justice,” and the word “righteousness” appears...

This is Exodus chapter 20, verses 2-14 from the 2023 English version from the Jewish Publication Society. It corresponds to what is called in Hebrew “The Ten Statements” and in Christian tradition “The Ten Commandments.” Perhaps Moses wrote this in the 15th century BCE, or perhaps others wrote it in the 6th and 5th centuries BCE.

A beautiful pattern

Emmy started growing flowers at home, using special lights and soil. She keeps all the flowers she grows, pressing them inside a large book to preserve them. When she remembers, at the end of the month she records the number of flowers she's grown since she started.

| at end of month | total flowers grown |
|-----------------|---------------------|
| 3               | 6                   |
| 5               | 10                  |
| 7               | 14                  |

If things keep going this way, how many flowers will Emmy have after twelve months? How many ways can you find the answer?

Method one: Multiply month three by four

Twelve is four times three. At the end of month three, there were six flowers. Multiply six by four and get 24 flowers at the end of month twelve.

Method two: Add months five and seven

Twelve is five plus seven. At the end of month five there were 10 flowers, and at the end of month seven there were 14. Add 10 and 14 and get 24 flowers at the end of month twelve.

Method three: Multiply twelve by two

The number of flowers is always double the month number. Multiply twelve by two and get 24 flowers at the end of month twelve.

That multiplier, two, goes by many names. In this case it is a singular value and arguably also an eigenvalue—though these are not always the same, and there's more to say.

Superposition

When these methods work, the pattern satisfies superposition. Superposition refers to putting multiple things “in the same place,” adding them up. In a somewhat special sense of linear, a system with superposition is also called linear. It might be best to think of linear as short for linear superposition. Linear Algebra is the mathematics of superposition, the mathematics of linear systems in this sense.

Superposition is usually defined based on method one (multiplying by a number works) and method two (adding things together works). Method three follows from those two. Much of Linear Algebra is about extending method three to more complicated cases.

You may have noticed that many patterns don't appear to follow superposition. Why does Linear Algebra focus on such simple patterns?

One reason is that by keeping the patterns simple in this way, it's possible to make them more complex in another way: by adding dimensions. Linear Algebra extends to many dimensions, becoming the math of data.

Another reason is that although the rules of linear superposition seem like a strong constraint on patterns, lots of complex patterns can still be represented or at least approximated.

Ellenberg is talking about geometry, and also poetry, and mentions “Euclid alone has looked on beauty bare” on page 7.

What Lincoln took from Euclid was the idea that, if you were careful, you could erect a tall, rock-solid building of belief and agreement by rigorous deductive steps, story by story, on a foundation of axioms no one could doubt: or, if you like, truths one holds to be self-evident. Whoever doesn't hold those truths to be self-evident is excluded from discussion. I hear the echoes of Euclid in Lincoln's most famous speech, the Gettysburg Address, where he characterizes the United States as “dedicated to the proposition that all men are created equal.” A “proposition” is the term Euclid uses for a fact that follows logically from the self-evident axioms, one you simply cannot rationally deny. (pages 12-13)

This is a little confused, since in the Declaration of Independence, “that all men are created equal” is held to be self-evident (an axiom) and not a proposition that follows from anything else. Either way, it's a better example of the complexity of meaning and interpretation than it is of logical reasoning.

Looking at the weathered books, you see that every controversy in education has been waged before, multiple times, and everything we think of as new and strange—math books like Inventional Geometry that ask students to come up with proofs on their own, math books that make problems “relevant” by relating them to students' everyday lives, math books designed to advance social causes, progressive or otherwise—is also old, and was thought of as strange at the time, and no doubt will be new and strange again in the future. (page 16)

This is neat and interesting to think about, especially vs. the present-oriented news and social media worlds...

Inventional Geometry is a series of questions for discovery learning. “It came out in 1860.”

The ultimate reason for teaching kids to write a proof is not that the world is full of proofs. It's that the world is full of non-proofs, and grown-ups need to know the difference. It's hard to settle for a non-proof once you've really familiarized yourself with the genuine article. (page 18)

There's a sense in which “the real world” (anything that isn't math) really doesn't have any proofs. The standards of mathematical proof are just not attainable outside abstract reasoning with explicit assumptions and inference rules. It isn't a question of better or worse reasoning, it's an epistemic difference that makes proof impossible. I think understanding this is a kind of maturity, but I don't think this is what Ellenberg is getting at.

I think Ellenberg is alluding to things like logical fallacies: the “bad arguments” which we so often perceive in the reasoning of people we disagree with. In good-faith discussion with people who share our assumptions, there is something to be said for this. But the scope is fairly narrow.

People don't disagree with us because they misapplied an agreed-upon inference rule and are just waiting for someone to correct their logic. They disagree with us because they have different assumptions and inference rules. Those are choices that are outside the realm of proof vs. non-proof. I think Ellenberg is being a little facile here. The topic deserves more.

On page 21 Ellenberg mentions the Alexander Horned Sphere, which took me a few minutes to understand with the help of Wikipedia. Infinite stuff is weird!

In geometry class we are usually not allowed to talk about picking up shapes and turning them over. But we ought to be. As abstract as we may try to make it, math is something we do with our body. Geometry most of all. Sometimes literally; every working mathematician has found themself drawing invisible figures with hand gestures, and at least one study has found that children asked to act out a geometric question with their body become more likely to arrive at the correct conclusion. (pages 30-31)

The study is Actions speak louder with words, and it's one of those that gets less interesting the more you look at it. The “children” were undergraduate students. The correct conclusions were so simple that it's astonishing any undergraduate could avoid the correct conclusions.

A lot of math is figuring out what we can, temporarily or for all time, get away with not caring about. (page 37)

Mathematicians before Poincaré, especially the Tuscan geometer and politician Enrico Betti, had wrestled with the question of assigning a shape a number of holes, but Poincaré was the first to grasp the issue that some holes could be combinations of others. And even Poincaré didn't really think about holes the way mathematicians do today; that would have to wait for the work of the German mathematician Emmy Noether in the mid-1920s. Noether introduced the notion of the homology group into topology, and it is her notion of “holes” we've been using ever since.

Noether expressed her ideas in the language of “chain complexes” and “homomorphisms,” not pants and milkshakes, but I'll stick with our current notation to avoid a wrenching stylistic shift. Noether's innovation was to see that it wasn't right to think of holes as discrete objects, but rather as something more like directions in space. (page 44)

Noether strikes again!

By the end of this chapter I'm still not entirely clear on how he's defining n-dimensional holes.

This might seem like a weird definition, but it makes everything work. (page 50)

This reminds me of the (uncharacteristic) bad introduction to matrix multiplication where Strang says “there is only one possible rule, and I am not sure who discovered it. It makes everything work.”

Making things work is good, but it isn't a satisfying reason for understanding.

Why take rigid motions as the fundamental symmetries? One good reason for this choice is that (though this is not easy to prove!) the rigid motions are exactly those things you can do to the plane that keep every line segment the same length—thus, symmetry, from the Greek for “with measure.” A better Greekism would be to use the phrase for “equal measure,” or isometry, and that is indeed what we call a rigid motion in modern math. (page 52)

“Perhaps too we shall have to construct an entirely new mechanics, which we can only just get a glimpse of, where, the inertia increasing with the velocity, the velocity of light would be a limit beyond which it would be impossible to go. The ordinary, simpler mechanics would remain a first approximation since it would be valid for velocities that are not too great, so that the old dynamics would be found in the new. We should have no reason to regret that we believed in the older principles, and indeed since the velocities that are too great for the old formulas will always be exceptional, the safest thing to do in practice would be to act as though we continued to believe in them. They are so useful that a place should be saved for them. To wish to banish them altogether would be to deprive oneself of a valuable weapon. I hasten to say, in closing, that we are not yet at that pass, and that nothing proves as yet that they will not come out of the fray victorious and intact.” (page 61, quoting Poincaré)

This was 1904, and motivated by concern about Maxwell's equations not being properly invariant under symmetries. Einstein is so often portrayed as if he came up with relativity in the wilderness, surprising everyone in 1905, but it really seems like the ideas were in the ether (sorry) around that time.

Page 68 is the first place Ellenberg mentions he was involved with the 2017 film Gifted. Later he gets into Bacon (and Erdős) numbers. His “Erdős-Bacon” number is 3+2=5 (page 315) which is neat.

It's a bit of a stretch, but if I'm allowed to enter the cinematic universe for doing the English subtitles for Scars then my Bacon number is 4, via Jung Hee-tae.

Apparently Pearson was quite a character! On page 75 Ellenberg mentions that he, among other things, invented the English word “sibling.”

Ellenberg also references one of his other books which has more about Pearson.

[Andrei Andreyevich Markov] wrote a lot of angry letters to the newspapers on social matters and was widely known as Neistovyj Andrei, “Andrei the Furious.” (page 85)

This is the Markov chain guy!

On pages 95-96, Ellenberg pokes some fun at writing coming out of GPT-3. Oh, the innocence of 2021!

Mathematics is a fundamentally imaginative enterprise, which draws on every cognitive and creative ability we have. (page 110)

A proof is crystallized thought. It takes that brilliant buoyant moment of “getting it” and fixes it to the page so we can contemplate it at leisure. More importantly, we can share it with other people, in whose mind it springs to life again. A proof is like one of those hardy microbial spores so robust they can survive a trip through outer space on a meteorite and colonize a new planet after impact. Proof makes insight portable. We mathematicians have been known to describe ourselves as standing on the shoulders of giants, but I prefer to say we're walking up a staircase made of the frozen thoughts of regular-sized people. We get to the top, we sprinkle our own thoughts on the ice, they freeze to the mass and make the staircase that much higher. Not as pithy but truer. (page 117)

This is very similar to How to bake pi: “I think that the key characteristic of proof is not its infallibility, but its sturdiness in transit.” (etc.)

I've been teaching math for more than twenty years now. When I started, I was driven by questions like this: What's the right way to teach a mathematical concept? Examples first, then explanation? Explanation followed by examples? Letting students discover principles by examining the examples I present, or stating principles at the blackboard and letting students discover examples? Wait, are blackboards even good?

I've come to feel there's no one right way. (Though there are certainly some wrong ways.) Different students are different and there is no One True Teaching Method that will ring everyone's saliva bell. ... Math teachers, I think, ought to adopt every teaching strategy they can, and shuttle through them in quick succession. That's the way to maximize the chance each student at least sometimes feels that their teacher is finally, after so much boring hoo-hah, talking about things in a way that makes sense. (page 122)

In my notes I called this idea “intentional diffusion.” Lots of entry-points. Maybe this is what Ellenberg's whole book is doing?

This phenomenon of computations we know exactly how to do, but don't have time to do, is a somber minor-key motif that sounds through the whole history of computer-age mathematics. (page 128)

The fastest computation is the one you don't do. (page 141)

For instance, there's a “2-adic” geometry of whole numbers in which the distance between two numbers is the reciprocal of the largest power of two dividing their difference. Seriously, this turns out to be a good idea. (page 191)

Huh! So that's what this p-adic business is?

The more dimensions you allow yourself, the better you can get the distances between points on your map to match the ones you've measured. Which means the data can tell you which dimension it “wants” to be in. (page 194)

He has a fun example just above, “In three dimensions, by the way, it's easy to make the distances between four points all the same; you place the four points at the corners of a shape called a regular tetrahedron.”

Anyway, this reminds me of Local Intrinsic Dimensionality (LID) which I think is a neat way to try to get dimensionality from distribution of data.

Ellenberg is probably thinking of multidimensional scaling (MDS) which cares directly about preserving distances between points. There's also of course PCA-style “how many dimensions explain some large percentage of variation?”

Hmm... LID probably relates with some directness to the MDS sense? In that the quicker we're running into more points, we get more points close to equidistant, we need more dimensions to preserve those? There's probably a way to formalize this... Ah especially if using geodesic distances/enforcing locality on MDS...

A really important and in some ways underpublicized fact about math is that math is very hard. (page 200)

If math class is easy, you're doing it wrong. (page 201)

“[T]he thoughts of pure mathematics are true, not approximate or doubtful; they may not be the most interesting or important of God's thoughts, but they are the only ones that we know exactly.” (page 214, quoting Hilda Hudson)

All working mathematicians experience mathematics as a kind of play, but Conway was singular in his insistence that play could be a kind of mathematics. (page 223)

If we know the ball's upward speed now, its upward speed a second later will be 16 meters per second less. For downward speed it's the opposite; the downward speed a second from now is 16 m/s more than it is right now. (page 238)

I think that's a typo? Or I'm missing something?

The section starting page 292, “The notes in the chord,” is a neat explainer on eigenvalues and Heisenberg uncertainty, different from others I've seen.

The example has infinite geometric sequences as state (he says Hilbert space in a footnote) and introduces a “shift” operator, which is the same as multiplying by the factor, hence it has that eigenvalue. He also has a “pitch” operator, multiplying each element by its zero-based position index. The only eigenvector then has zeros except for one position.

Then no non-zero vector is an eigenvector of both operators, and further if you shift then pitch, it's different from pitching then shifting. He shows the commutator shift-pitch minus pitch-shift, without calling it that.

Then it's an easy switch into position and momentum operators, and the commutator coming to the state multiplied by the reduced Planck constant. The positions and momenta you can measure are the eigenvalues of their operators.

The operators representing momentum and position do not commute. And the difference between the positionmomentum and momentumposition of a particle's state is just—well, not quite the state itself, but the state multiplied by a number called Planck's constant, and denoted ℏ. In particular, that means the difference can't be zero, which in turn implies, just as with the sequences, that a particle's state can never be an eigenstate for both the position and momentum operators. In other words, a particle can't have both a well-defined position and a well-defined momentum. In quantum mechanics we call this the Heisenberg uncertainty principle, and it walks around in a cloak of mystery and intrigue. But it's just eigenvalues. (page 297)

Ellenberg uses notes (C, D) as eigenvalues for the music example, which is kind of a neat way of talking about frequencies without using numbers... I've recently seen Heisenberg explanations talking about Fourier transforms, and Ellenberg mentions them toward the end of his piece here too. But Ellenberg's explanation seems to stand just fine without going into Fourier transforms. Certainly the ideas feel related... but Ellenberg's approach seems better, compared to just gesturing at Fourier transforms?

Can it be made even simpler?

A particle's state is two numbers (a, b). It is in the left position when (a, b) is a multiple of (1, 2) and it's in the right position when it's a multiple of (2, 1). It's moving up when (a, b) is a multiple of (2, 3) and it's moving down when it's a multiple of (3, 2). States like (3, 3) are not exact multiples; that one is equal share left and right, for example. No state is perfectly a position multiple and also a momentum multiple.

I think that isn't bad... It doesn't touch the commutator or the operators as things that modify state, but maybe that's okay... I feel like it's at least as good as the “wave in the general direction of Fourier transforms” explanation.

And if you want to learn about what I've left out, Sean Carroll's book Something Deeply Hidden is a great nontechnical primer in the mathematics underlying quantum physics. (page 297)

Possibly a good rec?

Around pages 335-336 he's talking about (sort of) the Flajolet–Martin algorithm, which is friends with LogLog and HyperLogLog, so it turns out I've used this thing a lot, and frequently been frustrated that the lazy machine wouldn't just do a real COUNT DISTINCT for goodness' sake.

The abstractions of geometry step in when our ability to draw gives out. (page 339)

Democratic governments are founded on the principle that every citizen's views are to be represented in the decision-making of the state. Like all good principles, this is easy to state, difficult to make precise, and almost impossible to implement in a fully satisfying way. (page 348)

“We are not trying to meet some abstract quota of definitions, theorems and proofs. The measure of our success is whether what we do enables people to understand and think more clearly and effectively about mathematics.” (quoting Bill Thurston, page 419)

“Basically, I'm not interested in doing research and I never have been,” [David Blackwell] said. “I'm interested in understanding, which is quite a different thing.” (page 419)

This was back in 2021, well before the recent AI math. Ellenberg holds up here.

Rows are horizontal and easy to read like lines of text.

Columns go up and down, like stone pillars holding up a building.

Columns are vertical and have things one on top of the other.

Here are three rows. How many columns?

\[ \begin{bmatrix} r & o & w \cr r & o & w \cr r & o & w \cr \end{bmatrix} \]

Here are three columns. How many rows?

\[ \begin{bmatrix} c & c & c \cr o & o & o \cr l & l & l \cr u & u & u \cr m & m & m \cr n & n & n \cr \end{bmatrix} \]

• Fairy Fun: Multiplayer open world game (2026)
• Vet game: Help some pets (2025)
• Typing game: Typing with flowers (2026)
• NICU tutor: Machines of the NICU (2026)
• NICUniversity: Welcome, first-year pre-med student. (2026)
• NICU Night Shift: A “simulator” game (2026)

Why pavement plots?

I was frustrated that there didn't seem to be a one-dimensional data visualization satisfying all five of these criteria:

• Simple: Explain it in one sentence
• One-dimensional: Easily stackable for comparing many distributions
• No distributional preconceptions: Not just central/unimodal data
• Minimal researcher degrees of freedom: Shouldn't depend on choices
• Handle large data sets: Elegant support for lots of data

I believe pavement plots satisfy all these criteria.

Why not histograms?

“The histogram is a poor method for comparing groups of univariate measurements.” (Cleveland, Visualizing Data, 1993)

Histograms are two-dimensional plots, in the first place. “Simple” is arguable, but certainly the researcher choices of bin position and bin width mean histograms can be misleading.

Why not Kernel Density Estimates (KDEs)?

KDEs are two-dimensional plots, and neither violin nor ridgeline variants fully overcome this. The concept is simple, but the details aren't. The researcher choice of bandwidth means that they can be almost as misleading as histograms.

Why not box plots?

Box plots are not quite one-dimensional plots, but they're close, which is why they can make for better comparisons between many distributions. They're close to simple, apart from the rules about whiskers and outliers. But box plots have a strong assumption that the data has central tendency and is unimodal. In all the years since the box plot, this doesn't seem to have really been addressed.

The core of a box plot, however, is the five-number summary: minimum, quartiles, and maximum. This is exactly what a pavement plot with four bins shows.

Why not rug plots?

Rug plots are pretty great, especially for small data sets with minimal repetition of values. They fail for large data, where they get impossible to read, and call for use of opacity and/or jittering, neither of which are ideal.

Rug plots can be basically identical to pavement plots for small data sets with particular choices of bin count, and for large data sets a pavement plot is equivalent to a rug plot with uniform “thinning” of the data.

More on pavement plots

Pavement plots still have some minimal researcher degrees of freedom, but they don't generally leave room for major misconceptions from a visualization.

There's the number of equal-data boxes to show, which gives flexibility in resolution. It's a choice; generally when there's enough data showing more bins is better.

There is also an underlying choice about how to calculate quantiles. Often it won't matter, but by all means use your preferred quantile method.

The material here roughly recapitulates what I presented in Pavement Plots: Less biased boxes and Pave the Planet. The Python implementation is on the Python Package Index, on GitHub, and has a documenting web site.

The author will stop at nothing to emphasize how smart von Neumann was. I am convinced, of course. So why did he not do anything quite so excellent as Einstein or Gödel? I suspect it may be to do with his having been rather too rich, and also (possibly in connection) that he spent too much time working on military things, often in secret. For whatever reason, it seems he didn't stick to things quite the way Einstein did.

I hadn't known that von Neumann was responsible for the ol' zero-is-the-empty-set construction, or that he's responsible for the quantification of utility based on gambling (in Theory of Games and Economic Behavior). He also seems to have been a sort of early example of techno-concern for the future along the lines of More Everything Forever, with his reproducing probes and so on. I think there was also a decent intro to Hilbert spaces in there.

An interesting detail has von Neumann's second wife Klára (Klári) writing the first code with a real subroutine, for the ENIAC just as it was converted to run stored programs, doing Monte Carlo with Metropolis. Neat!

“...mathematics is the foundation of all exact knowledge of natural phenomena.” (Hilbert)

I re-found this one and found another. Oh Hilbert! You know he wrote a geometry book? Hadn't been formal enough for him!

I was fascinated to learn about Emil du Bois-Reymond's “Die sieben Welträtsel” (seven world-riddles) of which he said "Ignoramus et ignorabimus" — "we do not know and we will not know." It was maybe just three that were "ignorabimus," but anyway the interesting point here is that Hilbert hated it, saying "Wir müssen wissen. Wir werden wissen." (We must know. We will know.) In the book this is largely setting up that Gödel was proving Hilbert wrong, but the whole thing is interesting to me because I've had related thoughts. In my shabby notes I even (quite by coincidence) used a list of seven.

Interesting stuff on Heisenberg's matrix mechanics... Apparently Heisenberg didn't even know what matrices are and sort of invented them in order to make his theory? He had to have Born tell him about matrices?

So Heisenberg uncertainty is non-commutativity of position and momentum matrices (operators)... I think Ellenberg has a nice toy example of this in his Shape...

So does Hilbert’s spectral theorem actually do spectra of atoms?! Eigenfunctions/values of Heisenberg matrices? It seems yes, but I don't have a complete handle on it yet...

“If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is.” (von Neumann)

I re-found that quote, together with another one:

“In mathematics you don't understand things. You just get used to them.” (von Neumann)

Surely you're joking, Mr. von Neumann? But I suppose he was something of a “shut up and calculate” quantum guy... And the guy whose broken “proof” arguably set back thought on quantum foundations for quite some time.

The AI Collective is a non-profit that hosts events. It doesn't totally make sense to me, but they seem to enjoy doing what they do, and the events can be fun.

They have two lists of values, or maybe one list with two parts. One is for the "community," which I think means anyone who comes to their events. The other is for the "team," which seems to be the staff and organizers—the people running the AI Collective. It's an interesting divide, somewhat like the divide between management and non-management, or capital and labor.

If you are given two numbers and asked to report a single number to describe them both, it's natural to choose the number exactly in between.

To get the number in between, you might subtract from the larger number while at the same time adding the same amount to the smaller number, until the numbers are equal.

You could imagine this physically as sliding along a number line from each number to their midpoint, or moving blocks until you have stacks of equal height.

To average four and two, you take one from four and add it to two, producing threes. You can picture this as starting from the point (4, 2) and finding the nearest point with equal values, by decreasing the first value while increasing the second, arriving at (3, 3).

The direction you moved is perpendicular to the line of equal values, which means you found the projection of (4, 2) onto that line. We can calculate that projection using any non-zero point on the line. What multiple of (1, 1), for example, is nearest (4, 2)?

With \( y = (4, 2) \) and \( x = (1, 1) \), the projection is \( x \) times a multiplier \( b \). Writing the calculation in the “normal equation” style, with column vectors,

\[ b = (x^Tx)^{-1}x^Ty = (1 \cdot 1 + 1 \cdot 1)^{-1} (1 \cdot 4 + 1 \cdot 2) = 3 \]

so we can find (3, 3) as \( 3x \)—and with \( x \) as ones, \( b \) also tells us the mean directly.

Our path to the line of equal values is the shortest such path from (4, 2). Calling it \( e \), notice that \( e \) is the hypotenuse of the right triangle with sides \( e_1 \) and \( e_2 \), which are the adjustments we made to the original numbers. By the Pythagorean theorem, \( e^2 = e_1^2 + e_2^2 \), so by minimizing \( e \) we have minimized the sum of the adjustments' squares. This is often called least squares.

Changing from item axes to dimension axes, we can plot \( y \) against \( x \). You can see that the mean is indeed exactly between the two numbers. The line of equal values has no presence here, except that the \( x \) we chose from it controls how far to the right we draw our points.

The line shown visualizes \( y = bx \). We had only been thinking of the case when \( x = 1 \), but the line extends, going through the origin.

The current view can also be where we start. We could have any \( (x, y) \) points we like.

Going back to item axes, the situation is just as when we were averaging, but now the line we're moving to isn't a line of equal values, and so the minimizing perpendicular also has a different slope.

We still calculate this \( b \) as we did before.

\[ b = (x^Tx)^{-1}x^Ty = (1 \cdot 1 + 3 \cdot 3)^{-1} (1 \cdot 2 + 3 \cdot 1) = \tfrac{1}{2} \]

We've fit a line through the origin that minimizes the squared errors between our line and the original items.

You can add more items (more points) and calculate as before, but that makes too many item axes to draw in two dimensions. (If you just wanted to average more than two items, you can do it, now with multiple justifications for adding up the numbers and dividing by how many you added!)

You can add more dimensions and calculate as before (with \( X \) a matrix and \(b \) a vector now) but the visualization of the solution can be harder to draw, and you'll have too many dimension axes to show in two dimensions.

Indeed, you can have most any number of items and dimensions and in general the “averaging machine” we've developed will handle it. Pretty great, for something that started by just shifting boxes around!

Tensors vs. n-dimensional arrays of numbers

Ever since TensorFlow, I've not quite understood what the heck a "tensor" is that's different from an n-dimensional array of numbers. Typical explanations are brief and hand-wavy, like "it's a physics thing, you wouldn't understand," or "it has to do with invariance, you wouldn't understand." These "explanations" are not helpful. With the reading of this book, combined with other investigations of linear algebra, I think I finally have it.

The invariance-under-transformation focus I find to be a bit of a red herring for thinking about the tensor-vs-array question. The thing is, if you start with an array, and you've chosen an appropriate transformation and measurement, then those guarantee invariance when you transform the array. The only way you can fail to comply with invariance is if you have two arrays (or a way for generating arrays) already and you find that they don't follow a given transformation.

The useful way to understand the relationship between tensors and arrays is that tensors are abstractions and arrays can be representations of tensors, once you choose a basis. This is exactly as, in linear algebra, a linear transformation is not a matrix, but a matrix can be a representation for a linear transformation, given a basis.

Every array of numbers can be a representation of some tensor in some basis, and the array alone doesn't tell you which tensor and basis, or even which type of tensor it might represent. For example, bilinear forms and linear transformations are both tensors and both can be represented by matrices: the matrix alone doesn't tell you which it is, if indeed it's intended to be either. (Specifying a transformation rule would specify the type of tensor, so invariance does connect here.)

A simple analogy might be that "a number is not a length." A number certainly can represent a length, and indeed any number can, but you have to know that the number is intended to represent a length, and ideally what the units of measurement are, if you want to really connect the number to a particular length.

TensorFlow works with n-dimensional arrays of numbers, regardless of whether they're representing a tensor at all.

I highlighted so much in the ebook that it warned me when I was exporting that I had used some percentage of the total 10% allowable by the publisher... Many of those selections follow, sometimes with my notes.

The shapes produced in these “vector files” do “carry” lines from point to point along the shape, and “carrier” is the original meaning of “vector.” (page xxv)

Kind of fun... "Carrier" of numbers?

But William Rowan Hamilton didn’t get so excited just because you can write down any number of components to represent various physical, digital, and economic systems or scenarios. Storing and representing data has been important ever since humans could count and write, but so has computation—and vectors and tensors enable you to do both the representing and the calculating at the same time. That’s where the magic comes into it: despite the simplicity and utility of writing vectors as lists of components, what Hamilton was so thrilled about was that when you consider the vector as a whole, not just its individual components, the rules of vector arithmetic make vectors far more powerful than numbers alone. (page xxv)

Hamilton pulled the rug out from almost everyone when he discovered that, unlike ordinary arithmetic, there is more than one way to carry out multiplications in vector arithmetic. (xxvi)

Photons are not Newtonian-style material particles, but in their interactions with atoms they can behave in a particle-like way. (page 19)

Interesting way of phrasing... Reminds me of What is real?.

So Hamilton asked his experimentalist friend Humphrey Lloyd to see if he could find this new kind of [conical] refraction—and when he did, Hamilton became a sensation, for it was perhaps the first time that mathematics had been used to predict the very existence of a physical phenomenon, rather than explaining what was already known. Many more such breakthroughs were to come: the prediction of the existence of Neptune would follow a decade later, and then a host of predictions, from radio waves and radiation pressure to photons and E=mc² to the Higgs boson, gravitational waves, and much, much more. (page 20)

Says this conical refraction is used for "optical tweezers"!

“Mathematicks were not, at that time, looked upon as Academical Learning, but the business of Traders, Merchants, Sea-men, Carpenters, land-measurers, or the like....” (page 33, quoting Wallis)

"That time" being his lifetime, in the sixteen-hundreds, I believe...

It was just six years after his appointment at Oxford when Wallis published his algebraic Arithmetica Infinitorum. At the time, most mathematicians still preferred using tangible geometric ideas to develop ways of handling infinite sums and infinitesimal increments—and Thomas Hobbes was particularly vociferous on the subject. His fame today rests on his political philosophy—and his dour conclusion that life is “nasty, brutish, and short” without the protection of a state. But he also dabbled in mathematics, and he expressed this pro-geometrical sentiment by pronouncing Wallis’s Arithmetica Infinitorum a “scurvy book” and “a scab of symbols.” In fact, he denounced “the whole herd of them who apply their algebra to geometry.” He evidently was not a sunny person. (page 33)

Einstein explained in detail the importance of this achievement: “Before Newton,” he said, “there existed no self-contained system of physical causality [about] the deeper features of the empirical world.” For instance, he added, Kepler’s observation-based laws of planetary motion, including the elliptical shape of planetary trajectories, established how the planets move, but not why. That’s because “these laws are concerned with the movement as a whole, and not with the question how the state of motion of a system gives rise to that which immediately follows it in time; they are, as we should say now, integral and not differential laws.” We’ll see Newton’s differential second law of motion shortly, and later we’ll see that this same preference for differential over integral laws is key to James Clerk Maxwell’s success in uncovering the secret of light. (page 35)

Among Newton’s early champions in France were Émilie du Châtelet and her partner, the provocative playwright Voltaire. In the 1730s, they collaborated on a popular book aimed at bringing Newton’s theories of gravity and light to a wider audience. (page 37)

Seems like this is "Éléments de la philosophie de Newton," subtitled "mis à la portée de tout le monde" ("put within everyone's reach") (English, French wiki). Neat that the two of them were doing science popularizations!

But what has i to do with vectors? It hinges on the question of how to represent information—and the related question of what kind of numerical constructs qualify as objects of mathematical study, by obeying clear mathematical rules. (page 53)

[Euler's] work on [complex numbers] really began to come together in the 1740s, when he connected i with the “circular” functions sinθ and cosθ. (Bear with me here, because circles are related to rotations, and coming to grips with the algebra of rotations is what will lead Hamilton to create vectors. But first he will need to understand the connection between rotations and i.) (page 54)

Goes on to explain how Euler took cosθ² + sinθ² = 1 and factored it as (cosθ + isinθ)(cosθ - isinθ) = 1, then getting e^iθ = cosθ + isinθ, based on Taylor series, it seems.

Multiplication by an imaginary number does indeed seem to be nothing more than a simple rotation! So, the wonderful thing about the complex plane is that it shows just how imaginary numbers can be related to real ones: via geometric rotations. (page 59)

Hamilton’s multiplication of couples, with its jumbled-up mixture of multiplications, additions, and subtractions, was something very different. (page 61)

This is talking about Hamilton defining multiplication of ordered pairs to behave isomorphic with complex numbers. The matrices as complex numbers approach is analogous but leans on matrix multiplication.

Newton had likewise refused the [Anglican] oath [required for fellowships at Comabridge and Oxford], not because he was in favor of religious tolerance—far from it—but because he was a secret Arian, which meant he didn’t believe in the divinity of Jesus. (page 72)

The single [vector] symbol a, [Hamilton] said presciently, is “indicative of one (complex) thought.” He went on to elaborate this in terms of his mathematical philosophy of time, suggesting to De Morgan that these n numbers—where you can choose any value of n, depending on the problem—represent a chronological ordering of events, and therefore a “notion of cause and effect.” (page 74)

But you may not know that the bold type often used today for vectors is due to the idiosyncratic Englishman Oliver Heaviside. We’ll meet him properly later, along with the urbane American Josiah Willard Gibbs, who gave us the dot for scalar products (also now called “dot products”) and the cross for the vector (or cross) product. (page 78)

Hamilton is often described as “the liberator of algebra.” (page 80)

Today, for example, noncommutative algebras underpin quantum mechanics and general relativity. In quantum mechanics, Heisenberg's famous uncertainty principle says that you cannot precisely measure a quantum particle's position X and momentum P at the same time. But you can precisely measure first one of these attributes, say the position, and then the other. The trouble is that by the time you’ve made this second measurement, the position measurement you made first will no longer be quite so accurate, because subatomic particles are always moving or vibrating—so you’ll get a different answer if you measure it again. (page 80)

This, I think, is not quite right. It makes it seem like the problem is logistics. I don't think that's the problem.

In general relativity, noncommutativity gives a measure of the curvature of space-time, so we’ll revisit this when we come to tensors. Meantime, you might already be thinking that along with vectors, ordinary matrices offer a more familiar example of noncommutative multiplication. (page 81)

Mathematicians had, in fact, been representing information in arrays and tables for thousands of years—but they hadn’t been adding and multiplying these arrays as algebraic entities in themselves until Arthur Cayley spelled out how to do it, fifteen years after Hamilton announced his quaternion algebra. (page 81)

A Memoir on the Theory of Matrices, 1858

For Gaussian elimination doesn’t use matrix multiplication or addition. (page 83)

I think this is wrong, or at least should be considered wrong, if understanding it properly.

Cayley initially got the idea for multiplying any two compatible matrices—not just a matrix and a vector—when he was trying to find a neat, laborsaving way of representing multiple linear transformations. (page 85)

But it turns out that quaternions are more efficient than matrices in more complicated situations where you need to combine several rotations around different axes, in order to orient an object in a particular way or to drive it via a smooth sequence of rotations—a satellite, for instance, or an airplane, a robot, your mobile phone screen, or a computer animation. Quaternions are more compact than matrices, and they are faster, because they require fewer calculations and use less computer power. (page 92)

For instance, to adjust the yaw and pitch angles, you’d multiply the two relevant matrices—and if you count out all the row-by-column steps in multiplying two 3 × 3 matrices, you’ll find there twenty-seven multiplications and eighteen additions. But if these rotations are represented by quaternions instead of matrices and roll, pitch, yaw angles, it turns out that there are only sixteen multiplications and twelve additions. (page 93)

I wish there was more of an example for this... Maybe I'll try to do one.

It’s a quirk of quaternion rotation algebra that turns θ into 2θ. And it’s a quirk of quantum mechanics that although you can rotate an ordinary material thing, such as a ball or a planet, through 360° and get it back to its starting point, quantum math says that a particle such as an electron needs to “spin” through 720° to get back to its original state. (page 94)

This is neat and I don't think I totally get it yet... something about quaternions being a "double-cover" of the sphere...

The book goes into spin here, and somehow or other I found this reference to Goudsmit's The discovery of the electron spin. (Ah: it was in note 25, page 362.)

The German physicist Wolfgang Pauli had already made a start on the nonrelativistic mathematics of spin and had introduced three quantities now known as Pauli matrices, which Dirac adapted in his formulation. As Pauli spelled out explicitly, these three matrices—which help describe spin angular momentum components about the three spatial axes—relate to each other in exactly the same way as Hamilton’s unit quaternions i, j, k: multiply any two of them and you get plus or minus the third—the same rules Hamilton had carved on Broome Bridge eighty-five years earlier! (page 96)

Maybe this is the connection?

Seems like Grassmann is an interesting character and early inventor of vector space ideas... in "The Linear Theory of Extensions." (mentioned page 103)

Even John Herschel, who had been so excited when he heard Hamilton lecture on his new discovery back in 1847, wrote to him saying the [Quaternion] Lectures would “take any man a twelvemonth to read, and near a lifetime to digest.” (page 108)

“It is worth noting,” Leibniz said, “that notation facilitates discovery. This, in a most wonderful way, reduces the mind’s labors.” (page 111)

On Maxwell (with Tait) starting to use vector fields... (page 124)

It was such a difficult problem that, just as with Grassmann’s entry in the Leibniz centenary competition, Germain’s was the only essay submitted. (page 126)

Are there still essay competitions like this?

The idea of something that “operates” when a function is “inserted” will become especially important when we come to tensors. (page 127)

(Like differentiation is an operator.) ("Higher-order functions"?)

When Maxwell first set out on his electromagnetic journey, sorting out the math of it had become rather tricky. It had to do with the advantages and pitfalls of using analogies in science—in this case, between gravity, hydrodynamics, and electricity. I’ve mentioned already the mathematical parallels between gravity and static electricity, but hydrodynamics—the study of how fluids behave as a result of forces—was important because in those early days people tended to visualize electricity as a fluid. That’s why they used the word “current” to describe the flow of “charged fluid,” whatever it was: the electron wouldn’t be discovered until the very end of the nineteenth century. But Maxwell realized, as very few had done, that analogies could lead you astray—indeed, electricity turned out not to be a fluid at all. Still, he knew that initially, at least, this analogy had offered a timely and powerful way into the mathematics of the new phenomenon of electromagnetism. The challenge was to use the math without confusing it with the physical reality. (page 129)

Gauss was one of the pioneers in developing this math, so the laws relating the gravitational and electrical flux through a closed surface to the enclosed amount of mass and charge, respectively, are called “Gauss’s laws.” (page 131)

But Faraday reasoned that even when you’d taken all the other iron filings away, the force from the magnet was still acting in all the same places as it was before—even if you could no longer see its effects. In particular, there must be force acting all the way through the space between the magnet and the “test” filing—just as there’d been when all the filings were present. To describe this idea, Faraday came up with the concept of “lines of force,” in direct analogy with the lines of iron filings. It was as if the filings made manifest the invisible force emanating from the magnet, just as light makes a watermark visible. (page 135)

The field, whatever it was made of, mediated these changes through space—just as the field of wheat mediates a breeze rippling through it like a wave. (page 137)

I'm not sure how I feel about this analogy... Not great, I think?

Maxwell had to find the right equations for relating this changing magnetic flux to the amount of current induced. This is known today as “Faraday’s law.” Then there was “Ampère’s law,” the mathematical version of Øersted’s discovery, in terms of the magnetic force produced by a changing electric current. (page 138)

So, part of what I’ve been trying to show throughout this story is just how long it takes for ideas to develop and find their best form. (page 140)

Fourier had published the heat equation in 1822, and then used it in the founding document of climate science, his 1827 paper, “On the Temperatures of the Terrestrial Sphere and Interplanetary Space.” (page 142)

Tait also linked this symbol with a name, “nabla,” which his assistant William Robertson Smith suggested, because “nabla” is the Latin transliteration of the Greek word for an ancient Assyrian harp, which had an inverted triangle shape similar to ∇. (Later, Gibbs will replace Hamilton’s “nabla” with “del”—apparently because ∇ also resembles an upside-down version of the Greek letter “delta.” Both names are used today.) (page 143)

Tait had also been working on a chapter on kinematics for his Elementary Treatise on Quaternions, which he’d been writing at the same time. He told his readers that although quaternion calculus may be difficult to learn, it is worth mastering because it offers “the most extraordinary advantage to the advanced student, not alone as aiding him in the solution of complex questions, but as affording an invaluable mental discipline.” (page 148)

Maxwell mentioned Thomson, but he went on to say that Hamilton had provided an even more fundamental type of classification than analogy, by dividing physical quantities into scalars and vectors. This innovation was so fruitful, Maxwell said, that the importance of quaternions could only be compared with Descartes’s invention of coordinates. (page 151)

But while Thomson believed quaternions and vectors were just a shorthand for equations already worked out, Maxwell saw that “Quaternions, or the doctrine of Vectors, is a mathematical method, but it is a method of thinking”—it is not just a creative laborsaving device for calculations. (page 158)

Thomson was perhaps the most famous dissenter. He didn’t even accept Maxwell’s theory of electromagnetism in component form, likening it to mysticism—all that math, with no mechanical model explaining in concrete terms how electromagnetic waves propagate! (page 159)

For physics in the everyday 3-D world, Clifford, like Maxwell, found that the separate scalar and vector products were far more useful than the full quaternion product—and this was another step toward the break away from quaternions to modern vector analysis. (page 162)

As for Lewes, he was a philosopher, critic, biographer, and a marvelous raconteur. In 1873 he was finishing a new book, Problems of Life and Mind. It was a forensic, empiricist exploration of the possible biological basis of consciousness, part of “the new psychology” of the time, which was bringing a more modern scientific approach to long-standing debates on the “mind-body problem”—the contested relationship between consciousness (mind) and the brain (body). (page 164)

This computational use of potentials is often known today as “gauge theory”: the idea is to choose a “gauge”—an equation in terms of A and ϕ—that simplifies the calculations without changing the values of E and B. In other words, the electric and magnetic fields remain invariant under “gauge transformations”—analogous to the way a rotating ball looks the same no matter which way you turn it. Gauges are now used to simplify calculations not just in electromagnetism, but in quantum theory and relativity, too. (page 176)

Gibbs quietly got on with his own work—and in 1881 and 1884, he self-published parts 1 and 2 of his little book Elements of Vector Analysis. He wasn’t the first American to work on what he called “multiple algebras,” algebras whose symbols represent more than one number, so that “double algebras” were complex numbers, vectors were “triple algebras” (in 3-D space), quaternions were “quadruple algebras” because they included a scalar plus the three spatial vector components—and so on. (page 178)

Inspired by Hamilton’s discovery of a new algebra without the commutative law, the Harvard mathematics professor Benjamin Peirce had long been working on creating dozens of other new algebras. Later, his son Charles proved that the only algebras allowing division or the existence of inverses—and hence ways of solving equations and of reversing rotations—are those of real numbers (that is, traditional school algebra), complex numbers, and quaternions. (page 178)

Today this general form is often called an “inner product,” following Grassmann. By contrast, I’ve already mentioned that the vector (or cross) product is only defined in three dimensions, whereas Grassmann’s analogous “outer product” was general, and we’ll see this in more detail in chapter 11. (page 179)

In 1888, Gibbs sent a copy of his Vector Analysis to everyone he could think of—including Thomson, Tait, and Heaviside. Heaviside was pleased to find he was not the only one working on vectors—but Tait was infuriated! And by the 1890s, a none-too-genteel debate had erupted in the pages of Nature and other leading scientific magazines. The question was: Should whole-vectorial systems be used at all—the likes of Thomson and Arthur Cayley said no, components were sufficient; but if they were to be used, then was it Hamilton’s quaternions or Grassmann’s algebra? Or should it be Gibbs’s and Heaviside’s utilitarian vector analysis, which Tait saw as little more than a rip-off of Hamilton? (page 179)

In his preface McAulay went so far as to advocate that students should study quaternions rather than Cartesian geometry—adding that the only way to encourage students to study the subject would be to make it “pay,” as he tellingly put it, by including more exam questions on the subject. Not much has changed: most students, it seems, want to learn how to pass exams rather than how to think. Such are the pressures on our students, and the limitations of our education and employment policies. (page 184)

But the key point is not the details but the fact that when you change your coordinate frame, the two sets of coordinates are related by specific equations. This is key to the idea of tensors. (page 190)

Invariance, yes... It seems like this is really more a characteristic of the change of coordinates and "specific equations" rather than the tensors themselves as such... (Depends on some idea of what kind of tensor we're even talking about, which basis...)

Tait didn’t know it, but the coordinate-free, invariant way of writing equations that he championed, such as a ∙ b = 0, is the key link between vector analysis and tensor analysis. (page 194)

if a charge is at rest in one frame, a relatively moving observer will see a magnetic field, but an observer in the charge’s frame will not. (page 202)

Four-Dimensional Space-Time Needs Four-Dimensional Vector Analysis (page 205)

As Einstein later recalled, “We found that the mathematical methods for solving problem 1 lay ready in our hands in the absolute differential calculus of Ricci and Levi-Civita”—that is, in tensor calculus. (page 219)

Arthur Cayley, for example, wasn’t just the inventor of matrix theory, and a spirited opponent of vector analysis; he was also a pioneer in both invariant theory and n-D geometry—specifically, the geometry of a curved surface when it is projected onto flat Euclidean space, like a shadow on the ground. (page 230)

Riemann didn’t give a name to these three-and four-index quantities. They are essentially the components of what are now—thanks to Ricci and Levi-Civita—called the Christoffel symbols and the Riemann tensor, respectively. (page 237)

the point once again is simply that if something has indices, it isn’t necessarily a tensor; as I indicated, it needs to have other properties—notably invariance under linear coordinate transformations. (page 237)

the Riemann tensor, the bedrock of general relativity. (page 238)

When Chisholm did her PhD with Klein in the 1890s, she, too, would be struck not just by his brilliant mind, but also by the way he would encourage his students to have the confidence to “never be dull!” (page 242)

(So technically they are “tensor fields,” but as I mentioned in the previous chapter, less rigorously “tensors” will do nicely.) (page 244)

And instead of saying “tensor calculus,” Ricci named the calculus of his systems the “absolute differential calculus.” By “absolute” he meant unchanging, because the interesting thing about tensors is that they encode the idea of invariance. (page 244)

You can see the idea of tensor (or outer) products in figure 11.1—which also highlights the fact that vectors and matrices can be thought of as tensors. There’s a sophisticated mathematical reason for this, but for now it’s enough to notice that like tensors, their components are represented with indices—one index for vectors, two for matrices, as the caption to figure 11.1 spells out. (page 245)

Ordinary numbers are represented by a symbol such as a. If they represent quantities that don’t depend on coordinates—such as temperature—then they are scalars, and since scalars don’t change under coordinate changes, they are tensors. (page 246)

The “rank” is also called the “order” of the tensor, for this is what Ricci called it when he introduced the idea. It has to do with the number of transformation matrices needed to transform from one coordinate system to another, but it essentially corresponds to the number of indices, each one representing a different type of information. (If you’re familiar with matrix algebra, note that for matrices viewed as tensors, this is a different use of “rank” from that in linear algebra.) (page 246)

These kinds of multi-index constructions—these tensors, and their products—are so important in data science that Google named one of its machine-learning platforms TensorFlow, and there are various other programs and tools, such as Tensorlab and Tensorly. (page 248)

the ordinary matrix rules don’t allow you to multiply a 2 × 1 matrix by a 2 × 2 one at all, but the tensor product does: (page 249)

where ⊗ is the symbol for tensor products. (page 250)

To make the probabilities work, α and β are complex numbers. (page 251)

This doesn't feel like enough explanation to me...

Maybe "to make the probabilities work in any basis"?

So, you can see that it makes a difference whether you write your vector, your data, as a row or column. This is the kind of thing that might make math seem bizarre and contradictory—but it’s just this sort of detail that piques a creative mathematician’s curiosity. (page 253)

Hmm... Is it?

So, in the case of a matrix formed from a column vector times a row vector, the elements can be represented as u1v1, u1v2, u2v1, u2v2, and so on. Straightaway you can see, just by looking at the notation, that you are multiplying two different kinds of vector. Decades later, Dirac would apply this distinction via his bra and ket notation. (page 253)

Today the word “vector” in this context refers to column vectors, while row vectors are called “one-forms” or “dual vectors.” Early twentieth-century researchers coined these terms; the idea originates with Grassmann, who had used the term “complement” instead of “dual.” Bras are examples of one-forms. Back in the 1880s, Ricci called vectors and one-forms “contravariant vectors” and “covariant vectors,” respectively, and these names are also used today. (page 253)

Okay so contravariant are regular column vectors, and covariant are row vectors... Change of basis yup yup yup.

So, to earn the title of tensor, it’s not enough simply to put information into a list or array—the Mesopotamians were doing that sort of thing four thousand years ago. To be a tensor, the arrays have to obey certain rules, just as we saw with vectors and matrices in chapter 4. (page 254)

Yes yes, but this is misleading...

But the most important thing in math and physics is the ability of tensors to represent information invariantly—“ absolutely”—without spurious data coming from the choice of coordinates. (page 255)

This is why Ricci said that vectors are tensors: their components—like higher-order tensor components—transform in a specific way under a change of coordinates. And scalars are tensors because they are just numbers or numerical expressions that don’t depend on the coordinates at all—so they are automatically invariant under coordinate transformations. (Just to dot i’s, not all numbers are invariants or scalars—for instance, frequency depends on the relative motion of the observer, as exemplified in the Doppler effect that we’ll see in the next chapter. And if the Unruh effect is finally detected, it may even turn out that temperature is not exactly the coordinate-independent scalar I said it was in chap. 7 and fig. 11.1—although you’d have to be traveling close to the speed of light to detect one degree of temperature change.) (page 263)

Looks like it's more about acceleration than speed, but sure.

Similarly, multiplying a row vector (a covariant vector or one-form or dual vector) by a column vector (a contravariant vector) gives a scalar—the scalar product in Euclidean space. (page 264)

Tensor expressions such as v_μu^μ and T_μνh^μν are examples of the tensor operation called “contraction”—because when you set a pair of upstairs and downstairs indices equal, you’re reducing, or contracting, the rank of your tensor. (page 266)

In fact, if T is a metric tensor—which from now on, and cribbing from Einstein, I’ll denote by g, with components g_μν—then this particular inner product is, in fact, just what we’ve been used to calling the scalar product of u and v. For, as we've seen, the metric actually defines the scalar product. (page 268)

So what's going on here is we have two data vectors, coordinates of a point or whatever, and they're both contravariant, upstairs index, column vectors. Then the metric is a bilinear form, covariant, two downstairs indexes. And in this case it's just ones on the diagonal, to give the usual dot product. (And then things get weird when gravity "bends spacetime" by changing the metric directly...)

Grossmann may never have realized what a crucial role he had played in putting tensors on the map. But since 1975 his legacy in general relativity has been honored in the Marcel Grossmann Meetings, which, every three or four years, bring together researchers from all over the world to discuss the latest developments. And in honoring Grossmann, these meetings also honor—implicitly, at least—the mathematical brilliance of Ricci and Levi-Civita, and the genius of Einstein. (page 319)

Wasn't he "Grassmann" before?

Oh jeez... There was Hermann Grassmann (1809–1877) who's not the same person as Marcell Grassmann (1878–1936). Was I asleep?

Anyway, cool party.

"NLA" for "Numerical Linear Algebra" (page 323)

In the mid-1920s Paul Dirac provides theoretical support for spin in his relativistic theory of quantum mechanical electron behavior—he uses Pauli spin matrices to describe electron rotations, and Wolfgang Pauli had shown that the math of these has exactly the same structure as Hamilton’s quaternion rotations. In 1975, Tony Klein and Geoff Opat show that spin is physical, not just a mathematical analogy. (page 336)

Hamilton’s process, from negative numbers/ science of time to complex couples to quaternions, is explained in detail by Teun Koetsier, “Explanation in the Historiography of Mathematics: The Case of Hamilton’s Quaternions,” Studies in History and Philosophy of Science Part A 26, no. 4 (1995): 593–616. (page 355)

Huh; the "Argand plane" is just the complex plane...

Explaining that to rotate with quaternions you do "rotated = transforming * original * transforming-inverse-which-is-also-complex-conjugate"...

Geometrically, this U^-1 factor is needed to counteract an extraneous rotation that happens because it's taking place a 4-D hyperspace, but that is beyond my scope here. Algebraically we’re talking about the quaternion analog of a matrix similarity transformation. (page 360)

For an example of matrices giving gimbal lock, see Justin Wyss-Gallifent’s MATH431 lecture “Gimbal Lock,” November 3, 2021: http://www.math.umd.edu/~immortal/MATH431/book/ch_gimballock.pdf. (page 361)

Ernan McMullin, “The Origins of the Field Concept in Physics,” Physics in Perspective 4 (2002): 13–39 (esp. 14). (page 369)

Michael J. Crowe, A History of Vector Analysis (page 377)

Martin Rees offers solutions to these problems [of education challenge etc.] in If Science Is to Save Us (Cambridge: Polity Press, 2022). (page 378)

Maxwell had assumed charge was continuously distributed—hence the charge density term ρ and current density J in his equations; Lorentz showed that these densities are an approximation or average of the distribution of charges (points), and that Maxwell’s equations are singular at these points, but hold everywhere else. (page 380)

Bertram E. Schwarzbach], The Noether Theorems: Invariance and Conservation Laws in the Twentieth Century (page 381)

Seems like an interesting book!

Minkowski had called these particular two-index quantities “vectors of the second kind,” and Sommerfeld called them “six-vectors.” Today they are simply called tensors—in this case, antisymmetric second-rank or second-order tensors, where the “rank” or “order” refers to the number of indices on its components. (If these tensors are defined through space, rather than at one point, then technically they are tensor fields.) (page 382)

Note that some researchers have suggested that the Event Horizon Telescope’s (EHT’s) first direct image of a black hole could, in fact, be that of a gravitomagnetic monopole rather than a black hole; they have calculated parameters that would distinguish the two possibilities when future, more accurate EHT observations are made: M. Ghasemi-Noedi et al., “Investigating the Existence of Gravitomagnetic Monopole in M87*,” European Physics Journal C 81, no. 939 (2021); https://doi.org/10.1140/epjc/s10052-021-09696-3. (page 385)

Maxwell’s process is “additive”—you add the light from the three filters and project the image onto a screen. It is used today in slides and in TV and digital images. Printed images use the “subtractive” method (discovered after Maxwell paved the way), where the three colors are reflected from the pigment on the paper rather than transmitted through the filters/ layers of pixels to a screen. The three primary colors of the subtractive method are the “opposites” of Maxwell’s—they are cyan, magenta, and yellow. (page 387)

“... mathematics is not an empirical science, ...” (Von Neumann, The Mathematician)

“Mathematics is not a deductive science—that's a cliché.” (Halmos, I want to be a mathematician)

“Pure mathematics is, in its way, the poetry of logical ideas.” (Einstein, epitaph for Noether)

“... mathematics is the science of skillful operations with concepts and rules invented just for this purpose.” (Wigner, The Unreasonable Effectiveness of Mathematics in the Natural Sciences)

“Mathematics is a tool for reasoning.” (Feynman, The Relation of Mathematics to Physics)

“Mathematics is no longer knowledge but a game of formulae, ruled by certain conventions, which is very well comparable to the game of chess.” (Weyl, Current Epistemological Situation in Mathematics)

“Mathematics is the art of giving the same name to different things.” (Poincaré, Science et Methode)

“... mathematics cannot amount to anything more than an immense tautology.” (Poincaré, Science and Hypothesis)

“Mathematics is the extension of common sense by other means.” (Ellenberg, How Not to Be Wrong)

“Mathematics is the art of explanation.” (Lockhart, A Mathematician's Lament)

“Mathematics is the study of things that come out a certain way because there is no other way they could possibly be.” (Ellenberg, How Not to Be Wrong)

“[Mathematics is] about how things fit together.” (Kra, Inside Dynamical Systems and the Mathematics of Change)

“[Mathematics] is security. Certainty. Truth. Beauty. Insight. Structure. Architecture.” (Halmos, quoted in his MAA obituary)

“Mathematics is prestidigitation.” (Linderholm, Mathematics Made Difficult)

“Mathematics is the science of the infinite.” (Weyl)

“Mathematics is the study of anything that obeys the rules of logic, using the rules of logic.” (Cheng, How to bake π)

“Mathematics is easy, as long as you have the right definition of ‘easy’” (Cheng, How to bake π)

“The pursuit of mathematics is the process of working out exactly what is easy, and the process of making as many things easy as possible.” (Cheng, How to bake π)

“Mathematics is interesting in so far as it occupies our reasoning and inventive powers.” (Pólya, How to Solve It)

“Mathematics is about problems, and problems must be made the focus of a student’s mathematical life.” (Lockhart, A Mathematician's Lament)

“Mathematics is not life” (Cheng, How to bake π)

“In the end mathematics is simply about things that are true.” (Cheng, How to bake π)

“Mathematics is not settled.” (Ellenberg, How Not to Be Wrong)