Transcript of Hickey's "Simple Made Easy (2011)" talk

transcript.txt

00:04 All right, who's ready for some more category theory? You're all in the wrong room.
00:16 So this talk I hope seems deceptively obvious. One of the things that's great about this conferences,
00:27 this is a pretty cutting edge crowd. A lot of you are adopting new technologies. A lot of you
00:30 are doing functional programming and you may, you may be nodding saying, yeah, yeah, yeah through
00:36 parts of this. And if some of it's familiar then that's great. On the other hand, I think that
00:43 I would hope that you would come away from this talk with some tools you could use
00:48 to help conduct a similar kind of discussion to this talk
00:52 with other people that you're trying to convince to do the right thing.
00:58 So I'll start with an appeal to authority. "Simplicity is a prerequisite for reliability."
01:05 I certainly agree with this. I don't agree with everything Dijkstra said and I, and I
01:10 think he might've been very wrong about proof in particular, but I think he's right about this.
01:17 We need to build simple systems if we want to build good systems.
01:21 And I don't think we focus on enough on that.
01:25 I love word origins. They are tremendous fun. One of the, the reasons why they're
01:29 fun is because words eventually become come to mean whatever we all accept them to mean,
01:35 you know, whatever's commonly understood to be the meaning is what it means.
01:39 And it's often interesting to say, well, I wish I could, I wish we'd go back to what it really means
01:44 and use that. And I think there's a couple of words that I'm going to use in this talk
01:47 that I would love for you to come away knowing the origins of, and try to use
01:52 more precisely, especially when talking about software.
01:57 So the first word is "simple". And the roots of this word are "sim" and "plex", and that
02:03 means one fold or one braid or twist. And that characteristic about being about one literally
02:14 fold or twist, of course, one twist, Hey, what's one twist look like? No twists, right? Actually.
02:22 And it, the opposite of this word is complex, which means braided together or folded together.
02:31 Being able to think about software in terms of whether or not it's folded together, is sort of
02:34 the central point of this talk. The other word we frequently use interchangeably with "simple"
02:41 is the word "easy". And the derivation there is through a French word, and the last step of the
02:47 derivation is actually speculative but I bought it because it serves this talk really well.
02:54 And that is from the Latin word that is the root of "adjacent", and which means to lie near
03:01 and to be nearby. And the opposite is "hard". Of course, the root of hard has nothing to do
03:06 with lying near. It doesn't mean lie far away. It actually means like strong or tortuously so.
03:17 So if we want to try to apply simple to the kinds of work that we do
03:22 we're going to start with this concept of having one braid
03:26 and look at it in a few different dimensions. I thought it was interesting in Eric's
03:30 talk to talk about dimensions because it's definitely a big part of doing design work.
03:34 And so if we want to look for simple things, we want to look for things that have sort of
03:38 one of something they do, they have one role,
03:41 they fulfill one task or job they're about accomplishing sort of one objective.
03:48 They might be about one concept like security and sort of overlapping with that as they may,
03:58 they may be about a particular dimension of the problem that you're trying to solve.
04:02 The critical thing there though, is that when you're looking for something that's simple,
04:05 you want to see it have focus in these areas.
04:09 You don't want to see it combining things. On the other hand, we can't get too fixated about one.
04:16 In particular simple doesn't mean that there's only one of them.
04:21 Right. It also doesn't mean an interface that only has one operation.
04:27 So it's important to distinguish
04:32 cardinality, right? Counting things from actual interleaving what matters for simplicity is that
04:38 there's there's no, interleaving not that there's only one thing. And that's very important.
04:48 Okay. The other critical thing about simple as we've just described it is if something is
04:55 interleaved or not that's sort of an objective thing. We can probably go and look and see,
05:02 I don't see any connections. I don't see anywhere where this twists with something else.
05:06 So simple is actually an objective notion.
05:11 That's also very important in deciding the difference between simple, easy.
05:17 So let's look at easy. I think this notion of nearness is really,
05:21 really cool. In particular obviously there's many ways in which something can be near, right.
05:27 The sort of the physical notion of it being near right, is something you know, like right there.
05:34 And I think that's where the root of the word came from. You know, this is easy to obtain cause it's,
05:38 it's nearby. It's not in the next town. I don't have to take a horse or
05:41 whatever to go get to it. We don't have the same notion of physicality necessarily in our software.
05:46 But we do sort of have our own hard drive or our own tool.
05:51 Or sort of the ability to make things physically near by getting them through
05:56 things like installers and stuff like that. The second notion of nearness is something being near
06:02 to our understanding, right. Or in our current skillset. And I don't mean in this case near to
06:10 our understanding, meaning a capability, I mean, literally near something that we already know.
06:15 So the word, the word in this case is, is about being familiar.
06:21 I think that collectively we are infatuated with these two notions of easy. We are just so
06:29 self-involved in these two aspects it's hurting us tremendously.
06:34 All we care about is, you know, can I get this instantly and start running it in five seconds?
06:39 It could be this giant hairball that you got, but all you care is "can you get it?".
06:45 In addition, we're fixated on, oh, I can't, I can't read that.
06:51 I can't read German. Does that mean German is unreadable? No, I don't know German.
06:59 So you know, this, this sort of approach is, is definitely not helpful.
07:03 In particular, if you want everything to be familiar, you will never learn anything
07:07 new because it can't be significantly different from what you already know
07:10 and not drift away from the familiarity.
07:14 There's a third aspect of being easy that I don't think we think enough about. That's
07:20 going to become critical to this discussion, which is, which now is being near to our capabilities.
07:26 And we don't like to talk about this because it makes us uncomfortable because what kind of
07:30 capabilities are we're talking about? Like, if we're talking about easy in the case of violin
07:34 playing or piano playing or mountain climbing or something like that. Well, you know, I don't
07:41 personally feel bad if I don't play the violin well, because I don't play the violin at all.
07:47 But the work that we're in is conceptual work. So when we talked, start talking about
07:51 something being outside of our capability well, you know, it really starts trampling
07:56 on our, our egos in a big way. And so, you know, due to a combination of hubris
08:02 and insecurity we never really talk about whether or not something is outside of our capabilities.
08:08 It ends up that it's not so embarrassing after all, because we don't have tremendously divergent
08:15 abilities in that area. The last thing I want to say about easy
08:20 and the critical thing to distinguish it from simple is that easy is relative,
08:26 right? Playing the violin and reading German are really hard for me.
08:30 They're easy for other people, certain other people. So unlike simple, where we can go and
08:36 look for interleaving, look for braiding. Easy is always going to be, you know, easy for whom
08:41 or hard for whom. It's a relative term. The fact that we throw these things around sort
08:47 of casually saying, oh, I like to use that technology because it's simple.
08:50 And when I'm saying simple, I mean easy. And when I'm saying easy, I mean, because I already
08:54 know something that looks very much a like, that is how this whole thing degrades, and we
08:58 can never have an objective discussion about the qualities that matter to us in our software. So
09:07 what's one critical area where we, where we have to distinguish these two things and
09:14 and look at them from a perspective of them being easy and being simple?
09:19 It is, it has to do with constructs and artifacts,
09:22 right? We program with constructs, we have programming languages, we use particular libraries
09:27 and those things in and of themselves when we look at them, like when we look at the code,
09:33 we've right have certain characteristics in and of themselves, but we're in a business of art.
09:40 Right. We don't ship source code and the user doesn't look at our source code and say, okay, oh,
09:45 that's so pleasant, right? No, they run our software and they run it for a long
09:51 period of time. And over time we keep glomming more stuff on our software,
09:56 all that stuff, stuff, the running of it. The performance of it,
10:00 the ability to change it all is an attribute of the artifact, not the original construct.
10:07 But again, here, we still focus so much on our experience of the use of the construct. Look,
10:14 I only had to type 16 characters. Wow. That's great. No semicolon or things like that.
10:21 This whole notion of sort of programmer convenience. Again,
10:25 we are infatuated with it. Not, not to our benefit.
10:29 On the flip side, it gets even worse. Our employers are also infatuated with it, right.
10:36 Those first two meanings of easy. What do they mean. Right. If I can get another programmer in
10:41 here. Right. And they look at your source code and they think it's familiar. Right. And they already
10:51 know the toolkit, right. So it's near, at hand. They've always had the same tool in their toolkit.
10:55 They can read it. I can replace you. It's a breeze, especially if I ignore
11:00 the third notion of easy. Which is whether or not anybody could understand your code.
11:06 Right. Because they don't actually care about that. They just care if somebody can go sit in
11:09 your seat, start typing. So again, as sort of a business owners this sort of again,
11:16 the same kind of focus on those first two aspects of easy, cause it makes programmers replaceable.
11:22 So we're going to see, we're going to contrast this with the impacts of long-term use.
11:27 Right. What does it mean to use this long-term and uh, yeah, what what's there,
11:33 what's there as all the meat, right? Does the software do what it's supposed to do?
11:40 Is it of high quality? Can we rely on it doing what it's supposed to do?
11:43 Can we fix problems when they arise? And if we're given a new requirement,
11:48 can we change it? These things have nothing to do with the construct as we typed it in
11:55 or very little to do with it and have a lot to do with the attributes of the artifact.
12:02 We have to start assessing our constructs based around the artifacts,
12:06 not around the look and feel of the experience of typing it in or the cultural aspects of that.
12:14 So let's talk a little about bit about limits. Oh, look, it does move.
12:19 This is just supposed to sort of lull you into this state where everything I say seems true
12:27 because I can't use monads to do that.
12:36 This stuff is pretty simple logic, right? How can we possibly make things that are reliable
12:40 that we don't understand? Right. It's very, very difficult. I think Professor Sussman made a great
12:48 point saying there's going to be this trade-off right. As we make things more flexible and
12:53 extensible and dynamic in some possible futures for some kinds of systems we are going to make
12:59 a trade-off in our ability to, to understand their their behavior and make sure that they're correct.
13:06 But for the things that we want to understand and make sure are correct, we're going to be limited.
13:13 We're going to be limited in our understanding and our understanding is very limited,
13:16 right? There's the whole notion of, you know, how many, how many balls can you
13:18 keep in the air at the time? Or how many things can you keep in mind at a time?
13:21 It's a, it's a limited number and it's a very small number . Right. So we can only consider
13:26 a few things. And when things are intertwined together, we lose the ability to take them in
13:31 isolation. So if every time I think I pull out a new part of the software I need to comprehend
13:37 and it's attached to another thing I had to pull that other thing into my mind because I
13:40 can't think about the one without the other. That's the nature of them being intertwined.
13:45 So every intertwining is adding this burden and the burden is kind of combinatorial as to the
13:52 number of things that we can, we can consider. So fundamentally this complexity and by complexity,
13:58 I mean this braiding together of things is going to limit our ability to understand our
14:05 So so how do we change our software? Apparently I heard in the talk today that agile and extreme
14:15 programming have shown that refactoring and tests allow us to make a change with zero.
14:23 I never knew that. I still do not know that
14:28 because that's not actually a knowable thing. It's, that's phooey,
14:36 If you're going to change software, you're going to need to analyze
14:39 what it does and make decisions about what it ought to do. You know, I mean,
14:42 at least you're going to have to go and say, what is the impact of this potential change,
14:47 right. And what parts of the software do I need to go to, to affect the change?
14:54 And you know, I don't, I don't care if we're using XP or Agile or anything else you're not going to
14:59 get around the fact that if you can't reason about your program, you can't make these decisions.
15:06 But I do want to make clear here because a lot of people, as soon as they hear the word reason
15:09 about they're like, oh my God, are you saying that you have to be able to prove programs?
15:13 I am not. I don't believe in that. I don't think that's an objective.
15:16 I'm just talking about informal reasoning, the same kind of reasoning we use every day to
15:21 decide what we're going to do. We do not take out category theory and say, you know, we, we actually
15:28 can reason without it. Thank goodness. So what about, what about the other side?
15:34 Right? There's two things you do with the future of your software. One is you add new capabilities.
15:38 The other thing is you fix the ones you didn't get, you know, so, you know, done so well. And
15:43 I like to ask this question. What's true of every bug found in the field? Got written.
15:50 It got written. Yes. What's a more interesting fact about it.
15:56 It passed the type checker. What else did it do? It passed all the tests. Okay. So now what do
16:05 you do? Right. I think we're in this world. I'd like to call it guard rail programming. Right.
16:15 It's really, really sad. We're like, I can make change cause I have tests.
16:20 Right? Who does that? Who drives their car around banging against the door?
16:25 I'm glad I've got these guard rails because I'd never make it to to the show on time.
16:32 Right. And, and do the guard rails help you get to where you want to go?
16:39 Like the guard rails, like guide you places. No,
16:42 those guard rails everywhere. They don't, they don't point your car in any particular direction.
16:48 So again, we're going to need to be able to think about our program. It's going to be
16:52 critical. All of our guard rails will fall. We'll have failed us. We're going to have this problem.
16:57 We're going to need to be able to reason about our program say, well, you know what? I, because maybe
17:02 if it's not too complex, I'll be able to say, I know through ordinary logic it couldn't be in
17:09 this part of the program. It must be in that part. And let me go look there first, things like that.
17:15 Now, of course, everybody's going to start moaning, but I have all this speed. I'm agile
17:19 and fast. You know, this easy stuff is making my life good cause I have a lot of speed.
17:27 So what kind of runner can run as fast as they possibly can from the very start of a race, right?
17:36 Only somebody who runs really short races. Okay.
17:43 But of course we are programmers and we're smarter than runners apparently because we know how to fix
17:49 that problem. Right. We just fire the starting pistol, every a hundred and call it a new sprint.
18:03 I don't know why they haven't figured that out, but right. It's my contention based on experience
18:12 that if you ignore complexity, you will slow down. You will invariably slow down over the long haul.
18:19 Of course, if you are doing something that's really short term, you don't need any of this.
18:22 You could write it, you know, with ones and zero's and this is my really scientific graph.
18:28 You notice how none of the axes are, there's no numbers on it. Cause I just, I completely made
18:34 it up. It's a, it's an experiential graph. And what it shows is if you focus on ease
18:43 and ignore simplicity. So I'm not saying you can't try to do both. But if you focus on ease
18:49 you will be able to go as fast as possible from the beginning of the race, but no matter what
18:52 technology you use or sprints or firing pistols or whatever the complexity will eventually kill you.
18:58 It will kill you in a way that will make every sprint accomplish less.
19:02 Most sprints be about completely redoing things you've already done.
19:06 And the net effect is you're not moving forward in any significant way. Now, if you start by
19:11 focusing on simplicity, why can't you go as fast as possible, right at the beginning?
19:15 Okay. Right. Cause some, some tools that are simple are actually as easy to use as some tools
19:21 that are not, why can't you go as fast, then you have to think you have to actually apply
19:28 some simplicity, work to the problem before you start. And that's going to be maybe this ramp up.
19:36 So one of the problems I think we have is this conundrum that some things that are easy
19:41 actually are, are complex.
19:43 So let's look. There are a bunch of constructs that have complex artifacts that are very
19:50 succinctly described, right? Some of the things that are really dangerous to use are like so
19:55 simple to describe they're incredibly familiar, right? If you're coming from object orientation,
19:60 you're familiar with a lot of complex things they're very much available, right.
20:06 And they're easy to use. In fact, by all measures,
20:09 conventional measures, you would look at them and say, this is easy.
20:14 Right. But we don't care about that. Right. Again, the user's not looking at our software
20:19 and they don't actually care very much about how good a time we had when we were writing it.
20:24 Right. What they care about is what the program does and the, and if it works well,
20:30 it will, it will be related to whether or not the output of those constructs
20:35 were simple. In other words, what, what complexity do they yield?
20:38 When there is complexity there? We're going to call that incidental complaint.
20:44 Right. It wasn't part of what the user asked us to do. We chose the tool, it had some inherent
20:48 complexity in it. It's incidental to the problem. I didn't put the definition in here.
20:54 But incidental is Latin for "your fault".
21:02 and it is, I think you really have to ask yourself, you know,
21:05 are you programming with a loom? You know, you're having a great
21:09 time. You're throwing that shuttle back and forth and what's coming out the other side
21:12 is this knotted, you know, mess. I mean, it may look pretty, but you have this problem, right?
21:20 Where's the problem. The problem is the knitted castle, right? Do you want a knitted castle?
21:32 What benefits do we do we get from simplicity? We get ease of understanding,
21:35 right? That's sort of definitional. I contend we get ease of change and easier debugging. Other
21:43 benefits that come out of it that are sort of on the secondary level are increased flexibility.
21:48 And when we talk more about modularity and breaking things apart, we'll see where they,
21:52 where that falls like the ability to
21:57 change policies or move things around. Right. As we make things simpler, we get more independence
22:03 of decisions because they're not interleaved. So I can make a location decision that's orthogonal
22:10 from a like a performance decision. And I really do want to make, you know, ask the question
22:17 agilist or whatever it is is having a test suite and refactoring tools going to make
22:24 changing the knitted castle faster than changing the Lego castle. No way, completely unrelated.
22:35 Okay. So how do we make things easy?
22:37 Presumably, you know, the objective here is not to just bemoan the software crisis. Right?
22:43 So what can we do to make things easier? So we'll look at those
22:45 parts, those aspects of being easy. Again, there's a location aspect,
22:49 making something at hand, putting it in our tool kit. That's relatively simple.
22:53 Right? We just install it. Right. Maybe it's a little bit harder because we have
22:57 to get somebody to say, it's okay to use it. Then there's the aspect of how do I make it familiar?
23:03 Right. I may not have ever seen this before. That's a learning exercise.
23:07 I've got to, you'll get a book, go take a tutorial, have somebody to explain it to me.
23:11 Maybe try it out. Both these things are we're driving, we're driving. We install, we learn
23:17 where it's totally in our hands. Then we have this other part though. Right. Which is the mental
23:23 capability part. And does the part, that's always hard to talk about the mental capability part.
23:29 Maybe because the fact is we can learn more things.
23:34 We actually can't get much smarter. We're not going to move. We're not going to move
23:39 our brain closer to the complexity. We have to make things near by simplifying them.
23:45 The, but the truth here is not that that's like there are these super,
23:48 super you know, bright people who can do these amazing things and, and everybody else is stuck.
23:54 Because the juggling analogy is pretty close, right? The average juggler can do
23:59 three balls. The most amazing juggler in the world can do like nine balls or 12 or
24:06 something like that. They can't do 20 or a hundred. Right? All, we're all very limited,
24:13 you know, compared to the complexity we can create, we're all, you know,
24:17 statistically at the same point in our ability to understand it, which is not very good.
24:22 So we're going to have to bring things towards us and because we can only juggle so many balls,
24:27 you have to make a decision. How many of those balls
24:30 do you want to be incidental complexity and how many do you want to be? Right. How many
24:36 extra balls you won't have somebody throwing your balls that you have to try to incorporate in here?
24:39 Oh, use this tool. And you're like, whoa. You know, more, more stuff who wants to do that?
24:47 All right. So let's look at a, let's look at a fact.
24:54 So I've been on the other side of this complaint and I, I like it. We can look
24:60 at it really quickly. Only because it, it's not, this analysis has nothing to do with the
25:06 usage. This complexity analysis is just about the programmer experience, right. So parens are hard,
25:12 right? They're not at hand for most people who haven't otherwise used it.
25:16 And what does that mean? It means that like, they don't have an editor that knows how to do,
25:22 you know, paren matching or move stuff around structurally, or they have one and they've never
25:26 loaded the mode that makes that happen. Totally given. Right. It's. Nor is it familiar? I mean,
25:33 everybody's seen parentheses, but they haven't seen them on that side of the method!
25:39 I mean, it's just crazy,
25:50 but you know, I think this it's, this is your responsibility,
25:54 right? To fix these two things as a user, as a potential user,
25:58 you got to do this. But we could dig deeper. Let's look at the third thing. Did you actually give me
26:02 something that was simple. Is a language built all out of parens simple in the case I'm saying?
26:10 Is it free of interleaving and braiding? And the answer is no right. CommonLisp and Scheme
26:18 are not simple in this sense, in their use of parens because the use of parentheses in those
26:24 languages is overloaded, right? Parens wrap calls, they wrap grouping, they wrap data structures,
26:34 right? That overloading is a form of complexity by the definition of, you know, I gave you, right.
26:40 And so if you actually bothered to get your editor set up and learn that the
26:44 parenthesis goes on the other side of the verb this was still a valid complaint. Now,
26:50 of course, everybody was saying easy. It's hard. It's complex in that we're using these
26:54 words really weekly. Right. But it was hard for a couple of reasons you could solve.
26:59 And it was not simple for a reason that was the fault of the language designer,
27:03 which was that there was overloading. Right. And we can fix that. Right. Just add another data
27:09 structure. It doesn't make lists, not lisp to have more data structures, right. It's still a language
27:13 defined in terms of its own data structures, but having more data structures in play means that
27:19 we can get rid of this overloading in this case, which then makes it your fault again.
27:25 Right. Because now this, the simplicity is back in the construct.
27:29 And it's just a familiarity thing, which you can solve for yourself.
27:35 Okay. This is an old dig at list programmers. I'm not totally sure what the what he was talking
27:45 about. I believe it was a performance related thing that lispers would just, they cons'ed up
27:49 all this memory and they did all this evaluation and it was, it was a pig. LISP programs at that
27:55 time were, were LISP programs at that time were complete pigs relative to the hardware.
27:59 So they, you know, they knew the value of all these constructs, right? This dynamism,
28:03 dynamic nature, these things are all great. They are valuable. Right. But there was this
28:07 performance cost. I'd like to lift this whole phrase and apply it to all of us right now.
28:14 Right. As programmers, we are looking at all kinds of things.
28:17 And I just see it, you know, read hacker news or whatever. It's like, oh, look, this thing has
28:21 this benefit. Oh, great. I'm going to do that. Oh, but this has this benefit. Oh, that's cool.
28:26 Oh, that's awesome. You know, that's shorter. You never see in these discussions, was there a
28:33 trade off? Is there any downside, you know, is there anything bad that comes along with this?
28:38 Never nothing. It's just like, we look all for benefits, right? So as programmers now, I think
28:42 we're, we're looking all for benefits and we're not looking carefully enough at the byproducts.
28:49 So what's in your toolkit. I have you know, I have these two columns,
28:58 one says complexity, and one says, simplicity, the simplicity column just being simpler.
29:05 It doesn't mean that the things over there are purely simple.
29:09 I didn't label these things bad and good. I'm leaving your minds to do that.
29:22 So what, what things are complex and what are the simple replacements. I'm going to dig into
29:25 the details on these. So I won't actually explain why they're complex. We're going
29:28 to say state and objects are complex and values are simple and can replace them. In many cases,
29:36 I'm going to say methods are complex and functions are simple and namespaces are simple.
29:41 And the reason why methods are there because often the, the space of methods the class or whatever
29:48 is also a mini very poor namespace. vars are complex, and variables are complex.
29:57 Managed references are also complex, but they're still simpler.
30:03 Inheritance, switch statements, pattern matching are all complex and polymorphism ala carte is
30:10 simple. Okay. Now remember the meaning of simple, meaning of simple means unentangled. Right.
30:19 Not twisted together with something else. It doesn't mean I already know what it means. Right?
30:24 Simple. Does not mean "I already know what it means." Okay. Syntax is complex. Data is simple.
30:33 Imperative loops, fold, even which seems kind of higher level
30:37 still has some implications that tie two things together.
30:40 Whereas set functions are simpler. Actors are complex and cues are simpler.
30:47 ORM is complex and declarative data manipulation is simpler.
30:53 Okay. Even Eric said that in his talk, he said it really fast near the end.
31:00 "Oh yeah. And eventual consistency is really hard for programs."
31:08 Conditionals are complex in interesting ways and rules can be, can be simpler. And inconsistency
31:16 is very complex. It's almost definitionally complex, right? Because consistent means
31:20 to stand together. So inconsistent means to stand apart. And that means taking a set of
31:25 things that are standing apart and trying to think about them all at the same time.
31:29 It's inherently complex to do that. And anybody who's tried to use a system that's eventually
31:33 consistent knows that. Okay. So there's this really cool word called complect. I found it.
31:43 I love it. It means to interleave or entwine or braid.
31:47 Okay. I want to start talking about what we do to our software that makes it bad.
31:52 I don't want to say braid or entwine because it doesn't really have the good,
31:56 bad connotation that complect has. Complect is obviously bad. Right?
32:02 It happens to be an archaic word, but you know, there's no rules that say you can't
32:06 start using them again. So I'm going to use them for the rest of the talk.
32:10 So what do you know about "complect"? It's bad. Don't do it.
32:14 Right. This is where complexity comes from. Right? Complecting. That's very simple.
32:25 Right? And in particular, it's something that you want to avoid in the first place.
32:29 Right? Look at this diagram. Look at the first one. Look at the last one,
32:34 right? It's the same stuff in both those diagrams, the exact it's the same strips. What happened?
32:41 They got complected. And now it's hard to understand the bottom diagram from the top
32:48 one, but it's the same stuff. You're doing this all the time.
32:52 You can make a program, a hundred different ways.
32:54 Some of them it's just hanging there. It's all straight. You look at it. You say, oh,
32:57 I see it's four lines this program, right. Then you could type in four lines in another language
33:02 or with a different construct. And you end up with this knot. So you've got to take care of that.
33:08 So "complect" actually means to braid together. And "compose" means to place together.
33:16 And we know that right. Everybody keeps telling us what we want to do is make composable systems.
33:20 We just want to place things together, which is great. And I think there's no disagreement,
33:26 right? Composing simple components. Simple in that same respect is the way we write robust software.
33:35 So it's simple. Right? All we need to do is (everybody knows this I'm up here
33:41 just telling you stuff you know), we can make simple systems by making them modular. Right.
33:47 We're done. I'm like halfway through my talk. I don't even know if I'm going to finish.
33:50 It's so simple, right? This is it. This is the key. No, it's obviously not the key.
33:58 Right? Who has seen components that have this kind of characteristic?
34:04 I'll raise my hand twice because not enough people are raising their hands. It's ridiculous. Right?
34:08 What happens? You can write modular software with all kinds of interconnections between them,
34:13 right? They may not call each other, but they're completely complected.
34:19 Right. And we know how to solve this. It has nothing to do with the fact that there are
34:24 two things it has to do with what those two things are allowed to think about.
34:28 If you want to really anthropomorphize and what do we want to make things allowed to think about?
34:33 And only these things, some abstractions, I don't know if that's coming out that well,
34:39 that's a dashed white version of the top of the LEGO, right?
34:44 That's all we want to limit things to. Cause now the blue guy doesn't really know anything about
34:48 the yellow guy and the yellow guy doesn't really know anything about the blue guy
34:50 and they both become simple. So it's very, it's very important
34:55 that you don't associate simplicity with partitioning and stratification, right?
34:59 They don't. Right. They are enabled by it. If you make simple components,
35:04 you can horizontally separate them and you can vertically stratify them.
35:09 Right. But you can also do that with complex things and you're going to get no benefits.
35:14 And so I would encourage you to be particularly careful not to be fooled by code organization.
35:21 Right. There's tons of libraries that look, oh, look, there's different classes. They're
35:26 separate classes. They, you know, they call each other and sort of these nice ways,
35:29 right. Then you get out in the field and you're like,
35:31 oh my God, this thing presumes, that, that thing never returns to number 17. What is that?
35:40 Okay. I'm not going to get up here and tell you, state is awesome. I like state. I'm not
35:45 a functional whatever guy, whenever I'm going to say instead I did this and it sucks. Right.
35:53 I did years and years, C++ you know, He-Man stateful programming. It's, it's really not fun.
36:01 It's not good. It's, it's never simple. Having state in your program is never simple, right?
36:08 Because it has a fundamental complecting that goes on in its artifacts, right? It complects
36:14 value in time. You don't have the ability to get a value independent of,
36:18 of time, and sometimes not an ability to get a value in any proper sense at all. But again,
36:26 it's a great example. This is easy. It's totally familiar.
36:29 It's at hand, it's an, all the programming languages. This is so easy. This complexity
36:34 is so easy. Okay. And you can't get rid of it, everything. So I'll have I
36:40 have modularity. And that, that assignment statement is inside a method, right? Well,
36:45 if every time you call that method with the same arguments, you can get a different result.
36:48 Guess what happened? That complexity. It just leaked right out of there. It doesn't
36:52 matter that you can't see the variable, right? If the thing that's wrapping it is
36:56 stateful. And the thing that's wrapping that is still stateful. In other words,
36:59 by stateful, I mean, every time we ask it the same question, you get a different answer.
37:03 You have this complexity and it's like poison and it's like dropping, you know,
37:07 some, some dark liquid into this, to a vase. It's just going to end up all over the place. The only
37:13 time you can really get rid of it is when you put it inside something that's able to present
37:18 a functional interface on the outside, a true functional interface, same input, same output.
37:24 You can't mitigate it through the ordinary code organization things.
37:29 And note in particular, I didn't talk about concurrency here.
37:32 This is not about concurrency. This has nothing to do with concurrency. It's about your ability
37:37 to understand your program. Your program was out there, it's single-threaded it didn't work.
37:41 All the tests passed. It, made it through the type checker, figure out what happened,
37:46 right? If it's full of variables, where are you going to need to try that? Recreate the
37:52 state that was happening at the client when it went bad. Is that going to be easy? No.
38:02 But we fixed this, right? Your language, your new shiny language has something called var
38:07 or maybe it has refs or references.
38:11 None of these constructs make state simple. That's the first primary thing. I don't want
38:16 to say that even of Clojure's constructs, they do not make state simple in the case
38:19 I'm talking about in the, in the nature of simple I'm talking about
38:22 but they're not the same, right? They all do warn you when you have state.
38:26 And that's great. Most people who are using a language where immutability is
38:30 not the default, and you have to go out of your way to get it, finds that the programs
38:34 they end up writing have dramatically like orders of magnitude, less state than they
38:39 would otherwise. Cause they never needed all the other state in the first place.
38:43 So that's really great. But I will call out Clojure and Haskell's references as
38:48 being particularly superior in dealing with this because they
38:53 compose values and time. They're actually little constructs that do
38:56 two things. They have some abstraction over time and the ability to extract the value.
39:03 That's really important. Because that's, that's your path back to simplicity. I mean, if I have
39:09 a way to get out of this thing and get a value out, I can continue with my program after I've
39:13 passed that variable to somebody else or a reference to something that's going to
39:16 find the variable every time through the varying thing, I'm poisoning the rest of my system.
39:22 So you know, look at the var in your language and ask if it does the same,
39:26 the same thing. All right, let's see why things are complex. State. We already
39:31 talked about it complects everything it touches. objects, complect state,
39:38 identity and value. They mix these three things up in a way that you cannot extricate the parts.
39:44 Methods complect function and state ordinarily, right? In addition, in some languages they
39:51 complect namespaces right. Derive from two things in Java, they have the same name, method.
39:57 It doesn't work. syntax, interestingly
40:02 complects meaning and order often in a very unidirectional way.
40:07 Professor Sussman made the great point about data versus syntax and it, you know, it's super true.
40:14 I don't care how much you really love the syntax of your favorite language. It's inferior to data
40:20 in every way. Inheritance complects types, right? So it says these two types are complected,
40:26 that's what it means. Inheritance. Complecting. It's like it's,
40:30 it's definitional, right? Switching and matching.
40:35 They complect multiple pairs of who's going to do something and
40:40 what happens. Right. And they do it all in one place in a closed way. That's very bad.
40:48 Vars and variables, again, complect value in time, often in an inextricable way, you
40:54 can't obtain a value. We saw a picture during a keynote yesterday this amazing memory, right where
41:01 you could dereference uh, an address and get an object out. I want to get one of those computers.
41:08 Right. Have you ever used one of those computers? I can't get one. I called Apple
41:12 and they were like, no. The only thing you can never get out of a memory address is a word,
41:17 a scalar, right? The thing that was all derided, right? Recovering a composite
41:22 object from an address. It's not something computers do. None of the ones that we have.
41:27 So variables have the same problem. You can not recover a composite mutable thing
41:32 with, with one dereference. Loops and fold loops are pretty obviously. Complecting what
41:40 you're doing and how to do it. Fold is a little bit more subtle, right? Cause it seems like this
41:45 nice. Somebody else has taken care of it, but it does have this implication about the order
41:49 of things this left to right bit. Actors complect what's going to be done and who's going to do it.
42:01 Now professor Sussman said the, all these talks have acronyms and I couldn't actually
42:07 modify my slides in time. So object relational mapping has, oh my God complecting going on.
42:15 You can't even begin to talk about how, how bad it is.
42:18 Right. And, and, and you know, if you're going to do like, duals, right?
42:25 What's the dual of value, is it co-alue? What's a co-value? It's an inconsistent.
42:36 Who wants that. And conditionals, I think are interesting, right? This is,
42:39 this is sort of more cutting edge area. We have a bunch of sort of rules about what
42:42 our program's supposed to do. It's strewn all throughout the program.
42:49 Can we fix that? because that's complected with
42:51 the structure of the program and the organization of the program.
42:55 All right. So if you take away two things from this talk one would be the difference
43:02 between the words "simple" and "easy". The other, I would hope would be the fact that
43:08 we can create precisely the same programs we're creating right now with these tools of complexity
43:15 with dramatically drastically simpler tools.
43:19 Right. I did C++ for a long time, I did Java. I did C#. I know how to make big systems in those
43:26 languages. And I completely. You do not need all that complexity. You can write as sophisticated
43:33 a system with dramatically simpler tools, which means you're gonna be focusing on the system,
43:38 what is supposed to do instead of all the guck that falls out of the constructs you're using.
43:43 So I'd love to say the first step in getting a simpler life is to just choose simpler stuff,
43:50 right? So if you want values, usually you can get it. Most languages have something
43:54 like values final or, or val you know, lets you like declare something as being immutable.
44:01 You do want to find some persistent collections.
44:03 Cause the harder thing in a lot of languages is getting aggregates that are values,
44:09 right? You got to find a good library for that or use a language where that's the default.
44:17 Functions. Most languages have them. Thank goodness. If you don't know what they are, they're
44:21 like stateless methods. Namespaces is something you really need the language to do for you.
44:27 And unfortunately, it's not done very well in a lot of places. Data. Please,
44:34 programmers. We supposedly write data processing programs.
44:37 There's all these programs, they don't have any data in them. They have all these
44:40 constructs. We put around it and globbed on top of data. Data is actually really simple.
44:45 There's not, there's not a tremendous number of
44:47 variations in the central nature of data, right? There are maps. There are sets, there are
44:52 linear, sequential things. There's not a lot of other conceptual categories of data.
44:57 We create hundreds of thousands of variations, have nothing to do with the essence of the stuff
45:02 and make it hard to write programs that manipulate the essence of the stuff.
45:05 We should just manipulate the essence of the stuff it's not hard. It's simpler. Also
45:11 same thing for communications, right? Are we all not glad we don't use the Unix method of
45:17 communicating on the web? Any arbitrary command string can be the argument list for your program
45:25 and any arbitrary set of characters can come out the other end. Let's all write parsers.
45:31 No, I mean, it's, it's a problem. It's, it's a source of complexity,
45:35 right? So we can get rid of that. Just use data. The biggest thing, I think the most
45:39 desirable thing, the most esoteric, this is tough to get, but boy, when you have it,
45:44 your life is completely, totally different thing is polymorphism ala carte, right?
45:50 Clojure protocols and Haskell type classes. And, and constructs like that give you the ability
45:56 to independently say I have data structures. I have definitions of sets of functions and I
46:03 can connect them together. And those are three independent operations.
46:08 In other words the genericity is not tied to anything in particular. It's available ala carte.
46:14 I don't know of a lot of library solutions for languages that don't have it. Already talked about
46:18 managed references and how to get them. Maybe you can use Clojure's from different Java languages.
46:24 Set functions. You can get from libraries. Queues, you can get from libraries,
46:27 right? You don't need a special communication language.
46:30 You can get declarative data manipulation by using SQL or learning SQL finally,
46:38 or something like LINQ or something like datalog. I think these last couple of things are
46:42 harder, right? We don't have a lot of ways to do this. Well-integrated
46:46 with our languages. I think at currently, LINQ is an effort to do that. Rules, right?
46:52 Declarative rule systems, instead of embedding a bunch of conditionals in our raw language at
46:58 every point of decision, it's nice to sort of gather that stuff and put it over someplace
47:02 else. And you can get rules systems in libraries, or you can use languages like Prolog. If you want
47:08 consistency, you need to use transactions and you need to use values.
47:12 Okay. There are reasons why you might have to get off of this list,
47:16 but boy, there's no reason why you shouldn't start with it. Okay. There's a source of complexity.
47:24 That's really difficult to deal with and not your fault. Call it environmental complexity, right?
47:30 Our programs end up running on machines next to other programs next to other parts of themselves.
47:36 And they contend, they contend for stuff, right? Memory, CPU cycles, and things like that.
47:43 Everybody's contending for it. This is an inherent complexity. Inherent is Latin for "not your fault"
47:51 in the implementation space. And those is not part of the problem,
47:55 but it is part of the implementation, right?
47:56 You can't go back to the customer and say,
47:58 the thing you wanted is not good because I have GC problems. But did you see problems
48:03 and stuff like that? They, they come into play. There's not a lot of great solutions,
48:07 right? You can do segmentation. You can say, this is your memory. This is your memory.
48:10 This is your memory. This is your CPU and your CPU. But there's tremendous waste in that,
48:15 right? Because you pre allocate, you don't use everything, you don't have sort of dynamic nature.
48:20 But the problem I think we're facing, and it's not one for which I have a solution
48:23 at the moment, is that the policies around this stuff don't compose.
48:28 Right. If everybody says I'll just size my thread pool to be the number of
48:32 cores. Well, how many times can you do that in one program.
48:36 Not a lot and have it still work out. Yeah. So unfortunately a lot of things like that
48:43 splitting that stuff up and making an individual decision is not actually making things simpler.
48:49 It's making things complex because that's a decision that needs to be made
48:52 by someone who has better information. And I don't think we have a lot of good sources for,
48:59 organizing those decisions in, in single places, in our systems. This is a hugely long quote.
49:07 Basically it says programming is not about typing, it's about thinking. So the next phase here,
49:18 I gotta move a little bit quicker is how do we design simple things of our own? Right. So the
49:23 first part of making things simple is just to choose constructs that have simple artifacts,
49:28 but we have to write our own constructs sometimes. So how do we abstract for simplicity, right.
49:33 And abstract again, here's an actual definition, not made up one, means to draw something away.
49:39 And in particular it means to draw away from the physical nature of something. I do want to
49:45 distinguish this from the, sometimes people use this term really grossly to just mean
49:49 hiding stuff. That is not what abstraction is, and that's not going to help you in this space.
49:56 There's too you know, I can't totally explain how this is done. It's really the job of designing.
50:02 But one approach you can take is just to do who, what, when, where, why. Well,
50:05 you can just go through those things and sort of look at everything you're
50:07 deciding to do and say, what is the who aspect of this?
50:10 What is the, what aspect of it? This can help you take stuff apart.
50:15 The other thing is to maintain the approach that says, I don't know, I don't want to know. I once
50:21 said that so often during a C++ course I was teaching that one of the students made me a shirt.
50:27 It was a Booch diagram. Cause we didn't have, have a, whatever it is now, the unified one.
50:33 And every line just said that
50:36 that's what you want to do. You really just don't want to know. All right. So what is,
50:39 what? What is the operations? You know, what, what is, what is it, what we want to accomplish,
50:44 right. We're going to form abstractions by taking functions or more particularly sets of functions
50:50 and giving them names. In particular you're going to use whatever your language lets you use. Right?
50:56 So if you only have interfaces, you'll use that if you have protocols or type classes,
51:02 you'll use those. So all those things are in the category. The things you use to make sets
51:06 of functions that are going to be abstractions and sets of specifications of functions.
51:12 The point I'd like to get across today is just that they should be really small,
51:17 much smaller than what we typically see. Java interfaces are huge.
51:22 And the reason why they are huge is just because Java doesn't have union types. So
51:25 it's inconvenient to say this function takes, you know, something that does this and that, and that
51:31 you have to make a, this and that and that interface.
51:33 So we see these giant interfaces. And the thing with those giant interfaces is that's a lot
51:37 harder to break up those programs. So you're gonna represent them with your polymorphism constructs,
51:43 they're specifications, right? They're not actually the implementations.
51:48 They should only use values and other abstractions in their definitions.
51:51 So you're gonna define interfaces or whatever type classes that only take interfaces and type classes
51:56 or values and return them. And the biggest problem you have when you're doing this part of design is
52:02 if you complect this with how. Right. You can complect it with how by jamming them together
52:07 and saying here's just a concrete function instead of having an interface or here's a concrete class,
52:11 instead of having an interface, you can also complect it with how more subtly
52:15 by having some implication of the semantics of the function dictate how it
52:19 is done. And fold is an example of that.
52:25 Strictly separating what from how is the key to making how somebody else's problem,
52:30 right? If you've done this really well, you can,
52:32 you can pawn off the work of how on somebody else. You can say, database engine, you figure out how
52:36 to do this thing or a logic engine, you figure out how to search for this. I don't need to know.
52:41 Who is about like data or entities. These are the things that are abstractions are
52:45 going to be connected to eventually, depending on how your technology works,
52:49 you want to build components up from sub components in a sort of direct injection style,
52:53 right? You don't want to like hardwire what the sub-components are.
52:56 You want to as much as possible, take them as arguments because that's going to give you more
53:00 programmatic flexibility in how you build.
53:05 You should have probably many more sub-components than you have. So you want really much smaller
53:09 interfaces than you have and more sub-components than you probably are typically having, because
53:12 usually you have none and then maybe you have one when you decide, do I need to farm out policy?
53:17 Yes. If you go in saying this is a job and I've done, who, what, when, where, why? And I found
53:21 five components, don't feel bad. That's great. You're winning massively by doing that,
53:27 you know, split out policy and stuff like that. And the thing that you have to be
53:32 aware of when you're building, you know, the definition of a thing from sub-components
53:36 is any, any of those kinds of, you know, red, yellow, thinking about blue, blue,
53:39 thinking about yellow, kind of hidden detail dependencies. So you want to avoid that.
53:46 How things happen. This is the actual implementation code, the work of,
53:49 of doing the job. You, you strictly want to connect these things together,
53:54 using those polymorphism constructs. That's the most powerful thing. Yeah you can use a switch
53:58 statement. You could use pattern matching, but it's glomming all this stuff together.
54:02 If you use one of these systems, you have an open polymorphism policy, and that is really powerful,
54:08 especially if it's runtime open, but even if it's not, it's better than nothing. And again, bewware
54:17 of obstructions that dictate how in some subtle way, because when you do that, you're really,
54:22 you're nailing the person down the line who has to do the implementation, you're tying their hands.
54:26 So the more declarative things are the better, the better things work. And the thing that
54:32 I mean, how is sort of the bottom, right? Don't mix this up with anything else. All
54:35 these implementations should be islands as much as possible. When and where, this is pretty simple.
54:42 I think you just have to strenuously, avoid comparing this with anything.
54:47 I see it accidentally coming in, mostly when people design systems with directly connected
54:53 objects. Right? So if you know, the, if your program is architected such that,
54:57 you know, this thing deals with the input, and then this thing has to do
55:02 the next part of the job. Well, if, if thing A calls thing B you just complected it, right?
55:09 And now you have a when and where thing, cause now
55:12 A has to know where B is in order to call B. And when that happens is whenever A, does it
55:20 right? Stick a queue in there, right? Queues are the way to just get rid of this problem. If you're
55:25 not using queues extensively, you should be, you should right away, like right after this talk.
55:32 And then this is the why part, this is sort of the policy and rules.
55:36 This is this is, I think this is hard for us. We typically put this stuff all over
55:41 our application. And if you ever have to talk to a customer about what the application does,
55:45 it's really difficult to sit with them in source code and look at it.
55:49 Now, if you have one of these pretend testing systems that lets you write English strings
55:52 so the customer can look at that. That's just silly. Right? You should have code that does
55:57 the work that somebody can look at, which means to try to, you know put this stuff someplace
56:02 outside, try to find a declarative system or a rule system which will let you you do this work.
56:08 Finally in this area information, it is simple,
56:12 right? The only thing you can possibly do with information is ruin it. Right? Don't do it. Right.
56:19 Don't do this stuff. We, I mean, we got it. Objects, objects just made to like encapsulate
56:24 IO devices. So there's a screen, but I can't like touch the screen, so I have an object, right?
56:28 There's a mouse. I can't touch the mouse's object. Right. That's all they're good for. They were
56:33 never supposed be applied to information and we apply them to information. That's it's just wrong.
56:38 It's wrong. But I can now say it's wrong for a reason, right? It's wrong. Cause it's complex.
56:45 In particular, it, it ruins your ability to build generic data manipulation things.
56:49 If you leave data alone, right. You can build things once that manipulate data and you can
56:56 reuse them all over the place and you know, they're right once and you're done. Yeah. The
57:01 other thing about it, which also applies to ORM is that it will tie your logic to representational
57:05 things, which again, tying complexing, intertwining, so represent data as data.
57:11 Please start using maps and sets directly. Don't feel like I have
57:15 to write a class now because I have a new piece of information. That's just silly.
57:21 So the final aspect, right? So we we've, we choose simple tools.
57:25 We write simple stuff and then sometimes we have to simplify other people's stuff. In particular,
57:31 we, we may have to simplify the problem space or some code that somebody else wrote.
57:36 This is a whole separate talk. I'm not going to get into right now.
57:39 But the job is essentially one of disentangling. Right. We know what's,
57:43 what's complex. It's entangled. So what do we need to do? Right. We need to somehow disentangle it.
57:50 Right? You're going to get this. You're going to need to first sort of figure out where it's going.
57:57 You're going to have to follow stuff around and eventually label everything. Right? This is the
58:02 start. This is roughly what the process is like. But again, it's a whole separate
58:06 talk to try to talk about simplification. All right, I'm going to wrap up a couple of slides.
58:14 The bottom line is simplicity is a choice. It's your fault if you don't have a simple system.
58:19 And, and I think we have a culture of complexity, to the extent we all
58:24 continue to use these tools that have complex outputs. We're just in a rut. We're just
58:29 self-reinforcing. And we have to get out of that rut. But again, like I said,
58:32 if you're already saying, I know this, I believe you, I already use something better.
58:36 I've already used that whole right column then hopefully this talk will
58:39 give you the basis for talking with somebody else who doesn't believe.
58:42 Right. Talk about simplicity versus complexity, right? But it is a choice,
58:48 right? It requires constant vigilance. We already saw the guard rails don't yield simplicity.
58:52 They don't really help us here. Right. It requires sensibilities and care.
58:58 Your sensibilities about simplicity being equal to ease of use are wrong. They're just simply
59:02 wrong. Right. We saw the definitions of simple and easy. They're completely different things,
59:08 right? So easy is not simple. You have to start developing sensibilities around entanglement.
59:14 That's what you have to just, you have to have entanglement radar,
59:17 right? You want to look at some software and say, ah, you know, not that I don't
59:20 like the names you used or the shape of the code, or there was a semi-colon. I mean,
59:25 that's also important too, but you want to start seeing complecting.
59:29 You want to start seeing interconnections between things that could be independent.
59:32 That's where you're going to get the most power. All the reliability tools you have,
59:37 right. Since they're not about simplicity, they're all secondary.
59:42 Right. They just do not touch the core of this problem.
59:47 Right. They're safety nets, but they're nothing more than that. So how do we make simplicity easy?
59:56 Right. We're going to choose constructs with simpler
60:03 artifacts, right? And avoid constructs that have complex artifacts.
60:08 It's the artifacts. It's not the authoring. As soon as you get in an argument with somebody
60:13 about, oh, we should, we should be using whatever, get that sorted out, you know,
60:17 because however they feel about the shape of the code, they type in is independent from this.
60:22 And this is the thing you have to live with.
60:25 We're going to try to create abstractions that have simplicity as a basis,
60:29 right? We're going to spend a little time up front simplifying things before we get started
60:36 and recognize that when you simplify things, you often end up with more things. Right.
60:42 Simplicity is not about counting, right? I'd have rather have more things hanging nice,
60:48 straight down, not twisted together then just a couple of things tied in and not.
60:54 And the beautiful thing about making them separate is you'll have a lot more ability to change it
60:58 which is where I think the benefits lie.
61:01 So I think this is a big deal and I hope everybody's able to bring it into practice or
61:09 use this as a tool for convincing somebody else to do that. So I'll leave you with this. This
61:16 is what you say when somebody tries to sell you a sophisticated type system. Thank you.