Video: The Coming Century of War Against Your Computer

00:00:01

welcome as as some of you know who come to these events we often run a short movie about long-term thinking before

00:00:16

these talks and this one's a bit of a stretch you know normally we do lots of time lapses and things like that but this talk is about privacy and I just

00:00:27

saw this movie with basically a $2,000 home-built drone that made me rethink privacy and I think it will for all of you enjoy [Music]

00:01:00

[Music]

00:01:43

[Music] [Applause]

00:02:02

[Music]

00:03:17

[Music] [Music]

00:04:40

you good evening I'm Stuart bran from the long now foundation I'm curious somebody here got here through Electronic Frontier Foundation the ffs bless your in its heart I've had

00:05:06

the honour of being one of the founding directors of EF f back when I was going through dramatic changes every six months or so seems to have settled down to major public service by now and so we

00:05:19

are happy to be co-sponsoring this between the long now foundation and EF F to bring Cory Doctorow here tonight you may have cards that are both replacing

00:05:32

my introduction to glory and also their space on the back for your questions which you may want to write during the event and send them out to people going up and down the aisles they will come down here to Alexander Rose who will get

00:05:46

the most embarrassing ones passed them to me and I will bring them up in the stage and hit Cory Doctorow with them you'll notice there's not much room for questions so short and legible in the

00:05:58

dark helps a lot talking about Corey I'm basically a fanboy I have been reading for the win and makers and brother and

00:06:13

because of those really got in touch with dr. oh now he is a PFF fellow which means he knows their secrets and he is

00:06:26

engaged one of the interesting things about a person writing contemporary science fiction which he does is not only are you getting a journalistic insight into things that are just showing the tips of their ears

00:06:39

so makers came out in 2009 and basically anticipated the entire maker movement that's now going on and goes beyond that to say what happens if the makers really win you get the victory condition and

00:06:54

everybody can make what happens then because things just keep rolling and then a good science fiction writer and a good political analyst will do that kind of thinking and writing and take

00:07:07

you there and so as a consequence usually when you know of writers political agenda they get pretty predictable and if you've read one book you don't need to read another because they're all the same story this is not

00:07:20

the case the Cory Doctorow he takes you to the place with his journalistic depth there is a political perspective that he plays out and he takes you there it's a

00:07:33

very interesting way to do politics he's not exactly telling stories tonight he's going dead at it tonight Cory Doctorow thanks folks so um this

00:07:51

talks a bit different than the talks that I've given I'm a career activist and so that means that generally speaking the end of my talks are supposed to end with a call to action lucky for me I've got an easy one every talk ins with you should join the FF you

00:08:04

should support e FF you should volunteer for you FF you should still do that but this talk doesn't end with a call to action although those are all good things to do because it's a it's a slightly different kind of talk it's a

00:08:16

talk where I don't really know how to solve the problem it's a talk about a problem that I don't know yet how to solve in the talk I will propose a technology and I want to make it clear that the technology I'm proposing is a

00:08:28

Gedanken experiment it's a thought experiment not a serious proposal not something that I think we should do but something to give you the idea of what I think we should do so with those two caveats out of the way I want to talk to you tonight about the coming civil war

00:08:41

over general-purpose computing so I gave this talk late in 2011 at 2083 the Kaos communications Congress in Berlin called the coming war on general-purpose computing in a nutshell the hypothesis

00:08:54

this talk was this computers in the Internet are everywhere the world is increasingly made of them we used to have separate categories of devices we had VCRs we had phones we had cars we had washing machines and now these are

00:09:07

all just computers with different boxes cars are computers we put our bodies into 747s our sub flying Solaris boxes with a bunch of SCADA controllers hearing aids

00:09:19

pacemakers other computer are other prostheses these are just computers we put in our body this means that all of the socio-political problems that crop up in the future are gonna have a computer in the middle of them and that means that every time a

00:09:32

problem crops up from now on you're gonna have a regulator saying what you know the entertainment industry has been saying for a while now make me a computer that can run every program we can compile except for this one program that pisses me off and the problem is

00:09:45

that we don't know how to make the general-purpose computer the Touring complete computer that's turing-complete minus one the computer that runs every program except for the program that scares someone we don't know how to make

00:09:57

a computer that will direct a self-driving car to do all the programs we can we can compile except for the drag racing program or the 3d printer to run all the programs we can compile except the one that lets us print out an

00:10:10

ar-15 or the thing that allows us to allows us to direct our bio scale assemblers to print out restricted compounds so we don't know how to make computers that can run all the programs we can compile except for the one that

00:10:23

pisses off a regulator or disrupts a business model or a bets a criminal the closest approximation we have for that device as a computer with spyware on it a computer that runs every program but has another program lurking on it in in

00:10:36

secret that watches everything you do to stop you from doing something that upsets someone a computer that watches everything you do and then if you do the wrong thing it intercedes and says I can't let you do that Dave a computer

00:10:50

that runs secret programs designed to be hidden from the owner of the device which the owner can't over override or kill in other words a computer running DRM now these computers are a bad idea for at least two significant reasons the

00:11:04

first is of course that they won't solve problems breaking DRM is not hard for bad guys the copyright wars lesson is that DRM is always broken with near immediacy because DRM can only work if

00:11:16

the I can't let you do that Dave program remains secret once the most sophisticated attackers in the world liberate that secret it becomes available to everyone else in the world now DRM the second reason is that DRM

00:11:29

has inherently weak security and makes all security overall weaker you can't be secure unless you can be certain about what software is running on your computer designing the I can't let you do that

00:11:41

Dave facility into a computer it creates the security vulnerability anyone who hijacks the facility can do things to your computer that it's designed not to let you find out about much less override and once government's solve

00:11:55

problems with DRM there's this perverse incentive to make it illegal to tell people things that might let them undermine DRM things like this is how the DRM works or here's a flaw in the DRM that would let an attacker or

00:12:08

secretly watch you through your webcam or listen through your microphone now I've had lots of feedback since giving this talk last year from various distinguished computer scientists technologists civil libertarians and

00:12:21

security researchers and I believe that within those fields there is a broad consensus that owners of computers all other things being equal should be able to control what runs on their computers that the world is a better place when

00:12:34

owners are in charge of what's going on in their computers now let's examine for a moment what it would mean to be an owner of a computer who's in charge of your computer now most computers today

00:12:46

are fitted with these things trusted platform modules or tpms now this is a secure coprocessor mounted on the motherboard of your computer the specifications of TPMS are published and

00:12:58

there's this industry body that certifies compliance with the specifications to the extent that the specification is a good one and the industry body is diligent in upholding it and enforcing its logo program it's

00:13:10

possible to be reasonably certain that you've got a real functional TPM in your computer and it faithfully implements the specification but the TPMS are secure in that they contain some secrets they contain some cryptographic keys

00:13:24

among other things but they're also secure in that they're designed to be tamper evident if you try to extract the keys from a TPM or change the TPM out or gimmick the TPM in some way it's

00:13:35

supposed to be obvious to the user that something's been done to her computer at least if your you know an average person and not like a sophisticated you know spy agency so one of the TPM threat models is that

00:13:47

crooks or governments or police forces or some other adversary might try to compromise your computer so the tamper evidence lets you know if someone's done something to your computer so that the TPM is no longer doing its job another

00:14:00

TPM model is that remotely someone might infect your computer with malicious software and once your computer gets infected with malicious software you're in great trouble all of the sensors that are attached to your computer the mic the camera the accelerometer the

00:14:14

fingerprint montt reader the GPS and so on might be switched on without your knowledge and then that data can be cached and sent to a bad guy or both all the data on your computer the sensitive

00:14:26

files your stored passwords and web history can be sent to a bad guy can be erased or can be changed all the keystrokes and your computers including your password can be logged and all the peripherals attached to your computer

00:14:38

can be covertly operated whether that's a printer a scanner a 3d printer a scatter controller a car the avionics system so on and so forth or subtly

00:14:49

altered and additional peripherals that may soon be connected to the computer that's infected include your optic nerve your cochlea the stumps of your legs and so on so you get the idea that there's a

00:15:02

lot at stake here and knowing what's going on in your computer and controlling what's going on in your computer now when your computer boots up the TPM can ask its bootloader the thing that boots up the operating system for

00:15:14

assigned hash of itself and verify that the signature from the hash comes from someone that the TPM trusts now once you trust the bootloader to faithfully perform its duties you can ask it to check the operating system and then check the ask the operating system to

00:15:28

check the programs you run and so on all the way up to whatever it is you're interacting with so that allows you to be to some extent or for some for some values or sure sure that you know what's running on your computer

00:15:40

and then any program is running in secret are running in secret because they've managed to do so by some by bootstrapping a flaw in the code and not by inserting something malicious into

00:15:52

the code that's a subtle difference but an important one they're doing so because not because your computer was designed to let them hide themselves from you but because they found a flaw in your computer's design now this always reminds me of

00:16:05

this guy Rene Descartes who starts off by saying how can I know about the world I don't know if my reason is true I don't know if my sense is faithfully report to me and you know some mental gymnastics and

00:16:17

he says all right I figured it out I know now that I can trust my senses I've done some logic work I trust my senses and I trust and from there from that nub of trust that I trust my senses and my

00:16:29

reason I can then work out a whole scaffolding from wit that I can that I can hang off of that nub of trust and on that scaffolding I can stand as I surveil the world and try to make sense

00:16:41

of it having that tiny nub of stable certainty is important it's from what it's the thing on which you can construct the stable edifice now a TPM can be a nub of stable certainty if it's

00:16:53

there it might reliably inform you about the code running on your computer now you may find it a bit weird to hear someone like me talking warmly about TPMS after all these are the technologies that make it possible to

00:17:06

lock phones and tablets and consoles and even some pcs so that they can't run the software that their owners want to run on them and when we talk about jailbreaking we usually means subverting a TPM why on earth would I ever want a

00:17:20

TPM in any computer that I owned well the Devils in the details as with everything important imagine for a moment two different ways of thinking about TPMS the first one we can call it lockdown and in the lockdown world your

00:17:32

TPM comes with a set of signing keys that it trusts and unless your bootloader is signed by the parties trusted by your TPM you can't run it and since the bootloader determines what operating systems you can run in the operating systems determine what

00:17:45

software you can run your computer is not under your control but now think for a moment about certainty you tell your TPM which keys you trust I trust EF F I trust WikiLeaks and I trust you bun two

00:17:59

and it tells you whether or not the bootloader is that it can find on your hard drive have been signed by those parties or you know or by you and it can faithfully report the signatures of any other bootloaders it finds and it lets you make up your own

00:18:12

damn mind about who you trust you trust yourself you trust other people you trust no one approximately speaking you can think of these two scenarios as corresponding to the way that ios and android work iOS

00:18:23

only lets you run the the operating systems that apple trusts and then the software that's built on top of them whereas android lets you run software that that you trust you can tick the

00:18:35

little box that says let me run someone else's software but there's a critical difference between what Android does and what I'm talking about tonight because Android lacks the facility to do that little bit of crypto work to let you know whether the software that you think

00:18:48

you're running is the software that you're running it's freedom but it's not certainty and a world where the computers that we're talking about can see and hear you where we insert our bodies into them and we insert them into

00:19:00

our bodies knowing what you are doing having certainty is a big deal which is why I'm coming around to the idea of a TPM assuming it's implemented in the certainty mode and not in the lockdown

00:19:13

mode now if that's not immediately clear think of it this way the war on general-purpose computing is what you get when the control freaks and government and industry demand the ability to remotely control your computer the defenders against that

00:19:26

attack people like me are also control freaks but we happen to believe that device owners should have control over their computers now both sides want control but we differ on which side gets control whose should be in control

00:19:39

control starts with knowledge if you want to be sure that songs can only be moved onto an iPod and not off of an iPod the iPod needs to know that the instructions being given to it by given

00:19:52

to it by a PC to which it's tethered are emanating from the actual Apple approved version of iTunes and not from something that impersonates iTunes in order to get it to do things that Gore Apple's aux

00:20:03

know if you want to be if you want to be sure that the PVR I've have hooked up to my TV won't record the watch wants video on demand programs you need to be sure that the tuner that's receiving it will

00:20:16

honor that flag and not output to any devices that it doesn't trust and if I want to be sure that you aren't watching me through my webcam covertly I need to know that the drivers are there

00:20:28

and that they're that they're faithfully honoring the convention that when the green light comes on the camera is on and when the cameras on the green light is on and if I want to be sure that you can't aren't capturing my passwords as I type them into my keyboard I need to

00:20:41

know that the OS isn't lying to me when it says there aren't any software key loggers on the system whether you want to be free or whether you want to enslave you need to have knowledge and control over the system so that's the

00:20:53

coming war on general purpose computing that's the talk I gave in Berlin now I want to investigate what happens in the event that we win the war on Germans computing that's the prospect I call becoming civil war over general-purpose

00:21:06

computers so I'll ask you now to stipulate this thought experiment that we went a victory on the freedom side and we have computers now that faithfully let owners know what's running on them and let owners choose

00:21:19

what's running on them the computers we get to control what's running on them because there's something on it that faithfully reports what's going on in them so there are two arguments I can make for why that would be a good thing and the first one is a human rights

00:21:33

argument if your world is made of computers then designing computers to override the owners wishes and decisions has significant human rights decisions today we're worried that the Iranian

00:21:46

government might ban from import any computer except for those that run operating systems that are that will allow them to spy on users but tomorrow imagine if the government that I live under the British government says that

00:21:58

from now on when the NHS gives you a cochlear implant it doesn't allow you to hear extremist speech or it takes any extremist speech that you hear and reports it back to the police or both the second reason for doing this is the

00:22:10

property rights reason the doctrine of first first sale it's a very important piece of consumer law it says that once you buy something you own something it belongs to you and you have the freedom to do anything you want even if it

00:22:23

pisses off the vendor now digital rights management opponents like me like saying things like if you bought it you own it property rights are a very very powerful argument have on your side especially in America

00:22:36

where there's strong property rights enforcement is often seen as the foundation of social remedies this goes double for this neighborhood for Silicon Valley you can't swing a cat for hitting someone who believes that the major or

00:22:49

only legitimate function of governments is to enforce property rights and the contracts around them which is to say that if you want to win a nerd fight property rights are a powerful weapon to have on your side and not just nerd

00:23:01

fights if you go to DC it's a great place to have it's a great thing to have on your side too and this is why people involved in in the copyright fight are so touchy about the term intellectual property which was brought in very cynically in the mid 70s to replace the

00:23:15

term of art of the day which was the intellectual monopoly or the creators monopoly creators monopoly is a hard thing to argue for going to a regulator or law maker and saying my monopoly isn't being forced vigorously enough is

00:23:27

a lot less palatable than my property rights are being trampled upon so this is this is where the civil rights part comes in the Civil War part comes in I beg your pardon

00:23:39

human rights and property rights both demand that computers not be designed for remote control from governments and corporations and so on that is owners should be allowed to specify the software they're running to freely

00:23:52

choose the nub of certainty from which they suspend the scaffold of their computers security now remember security is relative you are there's no such thing as abstract security you can only be secured from attack up from a certain

00:24:05

attack so I might be secured from my attack on my from an attack on my ability to freely use my music when if you can control my computing environment but if I can control my music and you

00:24:17

can't control my computing environment and you're the record industry then you are less secure and your ability to charge rent every time I want to listen to music if I get to choose the nub from which the scaffold dangles I get control

00:24:29

and power to secure myself against attackers now if the Recording Industry Association of America or the government or Monsanto get to choose that nub then they get the control and the power to secure themselves against me so let's

00:24:43

all agree at the very least that owners should be able to know and control the saw we're running on their computers but what about users users of computers don't always have the same interests as

00:24:54

owners of computers and increasingly we are all going to be the users of computers we don't own because we are going to inhabit a world made of computers where you come down on the conflicts between users and owners I

00:25:07

think will end up being one of the more meaningful questions of the coming century and there's no easy answer I know of for figuring out where not where to land given any city any situation no rule of thumb that works for everything

00:25:21

so let's start with a position we can call the property maximalist right here's here's black stone on property if it's my computer I should have the absolute right to dictate to users the

00:25:33

terms on which they can use it if you don't like the rules that I've set for my computer go use someone else's computer how does that work in practice well we can have some combination of of law and technology you get a computer

00:25:45

and out of the box it asks you to give it a password an administrative password and then when you've given it the administrator and all the administrator can choose who's operating systems they trust so no one else is allowed to come

00:25:57

and tell you what to trust and you can tell anyone who wants to use the computer which operating systems it's going to run so you generate a random signature and that signature or you

00:26:10

write generators that you have a signature that signatures used to generate a random key in that that belongs to you without that key no one can change the list of trusted parties now we could even make it against the law to subvert that system for the purposes of tricking people into running

00:26:22

software they didn't choose themselves that would make spyware extra super illegal and it would also make sneaking DRM onto people's computers illegal we can design this TPM so that if you

00:26:34

tamper with it or you remove it it becomes really obvious you know you give it a fragile housing so that anytime you you attack it it kind of breaks apart and anyone can tell that something funny is going on

00:26:47

now I can see lots of benefits to this but there unquestionably some downsides too so think for a minute of self-driving cars now there's a lot of these around already of course coming out of Google and elsewhere and it's easy to understand on the one hand

00:27:00

while I self-driving cars would be incredibly awesome because as a species we are terrible drivers seriously I mean cars kill the out of us it is the

00:27:11

number one cause of deaths for people in America aged 5 to 34 now I've been hit by a car and I've also cracked up a car and I'm perfectly willing to stipulate that human beings have no business

00:27:23

driving cars now it's also easy to understand how we might be nervous about people being too able to home brew their own firmware for self-driving cars on the one hand we'll want the source code

00:27:35

for cars to be widely published so that we can all scrutinize it and make sure it's good but on the other hand I think there's going to be a very plausible argument to say some authoritarian body gets to certify what's running on the

00:27:49

highways now we're back to the now we get back to this problem of whether or not you get to decide whether a computer you own the computer in your car you get to decide what it's doing now there's two problems with this solution the

00:28:01

first one of course is that it won't work as the copyright wars have shown us firmware locks aren't very effective against determined attackers who are sitting in front of them people who want to sow mayhem with their custom firmware

00:28:13

are going to be able to and what's more this is just not a good security approach if the hick Euler security models depend on all other vehicles being well behaved and the unexpected never arising we are all dead meat

00:28:28

self-driving cars need to be conservative in the approach and their approach to their own conduct and they need to be liberal in their expectation of other people's conduct it's the advice you got on day one a driver's ed and it's good advice whether a human

00:28:40

being is behind the wheel or a computer is but the second reason is is a more kind of philosophical and social one because the when you start to say well there are classes of devices that everyday people own that some remote

00:28:53

authorities should be able to override their decisions on it invites some pretty ugly parallels you remember the term information superhighway if we can justify securing the physical roads by demanding that the state or some state

00:29:05

like entity gets to certify the firmware of the devices that crews on its lanes how do we articulate a policy on explaining why the vices on our equally vital virtual highways with comparable with comparable

00:29:19

firmware locks for PCs and phones and tablets and other devices why those shouldn't be under state control after all we have a general-purpose network and that general-purpose 'no smiie pnes that MRIs spaceships air traffic control

00:29:31

systems and so on all share the information superhighway with your Gameboy your Arduino linked fart machine and the dodgy voyeur canceled by spammers from the Pearl River Delta and and think for a moment about a V onyx

00:29:45

and power station automation these are much trickier questions at the FAA mandates a certain firmware for 747s it's probably gonna want those 747s designs so that it and it alone controls

00:29:57

the signing keys for their boot loaders and just as the Nuclear Regulatory Commission will want the final say on the form we're running on reactor piles now this may be a problem for the same reason that modifying that a ban on

00:30:10

modifying car firmware will be a problem because it establishes the idea that the authorities should be allowed to control the firmware on some device that that belongs to you or some other private entity but might and maybe that

00:30:23

airplanes and nukes are so different from all the other things that we own that that wouldn't happen I mean there are already subject to such overweening regulation you know no-knock warrants and and surprise inspections that it may

00:30:36

be that adding another layer of regulatory control won't lead us to think well if you can control airplanes why not you're the computer on your desk but there's a there's a bigger problem with owner controls what about people

00:30:47

who use computers but don't own them this is not a group of people that the eight IT industry has had a lot of sympathy for on the whole we have devoted an enormous amount of energy to stopping non owning users from doing a

00:31:01

bunch of crazy things like inadvertently breaking the computers that they're using downloading menu bars typing random they find on the internet into their terminals plugging malware infected USB sticks in disabling

00:31:13

firewalls installing plugins adding repositories adding certificates to the machines of the browser's trust route punching holes on the network perimeter by accident or accidentally cross connecting the networks that are required to remain separate like the

00:31:26

sensitive internal work in the untrusted public network and to stop users from deliberately doing bad things like installing key loggers and spyware that they used to attack future users or misappropriating secrets

00:31:39

that are on the network or snooping on network traffic or deliberately breaking their machines or deliberately punching holes in the network's perimeter or deliberately disabling the firewall or deliberately interconnecting those networks that are supposed to remain

00:31:51

separate there's a kind of symmetry here DRM and it's cousins are deployed by companies who believe that you can't and shouldn't be able trusted to set policy on your own computer IT systems are

00:32:05

deployed by computer by computer owners who believe that computer users can't be trusted to set policies on the computers that they use now I'm a former systems administrator and a former CIO and I'm not going to pretend that users aren't a

00:32:19

challenge every day for people who managed IT systems but I think that there are good reasons to treat users as having rights to set policies on the computers that they don't own I want to start with the easy one the business

00:32:31

case for this when we demand freedom for owners we do so for lots of reasons but one important reason is that computer programmers just can't anticipate all the contingencies that their code might

00:32:43

run up against that when the computer says yes owners still might need to say no and when the computer says no owners still might need to say yes the idea that owners might possess situational

00:32:55

awareness that just can't be perfectly captured in a series of nested if then statements now this is where communism and libertarianism converge this guy Hayek thought that expertise was a

00:33:07

diffuse thing and that you were more likely to find the situational awareness good for good decision-making very close to the decision itself devolution gives you better results than centralization and then there's this guy who thought

00:33:21

that the believed in the legitimacy of workers claims over their working environment saying that cut that the contribution of labour was just as important as the contribution of capital and demanding that workers be treated as the light rightful owners of their

00:33:33

workplace with the power to set policy over it for totally opposite reasons these two believed that the people at the coalface should be given as much power as possible now the death of mainframes was attended by an awful lot of concerns

00:33:46

over users and what they might do to their enterprises in those days users were even more constrained by their IT system than they are today they could only see the screens that the mainframe let them see and only undertake the

00:33:59

mainframes that the mainframe let them undertake the operations that the mainframe let them undertake when the PC and VisiCalc and Lotus 1-2-3 appeared employees actually risked getting fired by bringing those machines into the

00:34:12

office or bringing home office data to run on them at home because they had a computing need that couldn't be met by the constraints set by their firms and by the its IT department and because

00:34:24

they didn't think that if they came to their IT department or their employers with their demands that they would be recognized as legitimate after all the standard response to something like that from a user is one or more of the following are regulatory compliance

00:34:37

prohibits you're doing the thing that will help you do your job better if you do your job that way we won't know if you've done it right you only think you want to do your job that way it's

00:34:49

impossible to make a computer do what you want it to do corporate policy prohibits doing your job that way now some or all of these may be true although they often aren't and even when

00:35:01

they are true they're the kind of truths that we give bright young geeks millions of dollars in venture capital to falsify while middle-aged administrative assistants merely get written up by HR

00:35:12

for trying to do the same thing the prom the personal computer arrived into the enterprise by the backdoor over the objections of the IT department without the knowledge of management at the risk

00:35:25

of censure and termination and it made the companies that fought at billions trillions the reason that giving workers more powerful more flexible tools was good for firms is that people are

00:35:38

generally smart and they generally want to do their jobs and because they know stuff that their bosses don't know as an owner you don't want the devices that you buy locked because you might want to do something that the designer didn't

00:35:51

anticipate and employees don't want the devices that use all day locked because they might want to do something that the IT department didn't anticipate this is the soul of Hayek ISM we're smarter at the

00:36:04

edge than we are in the middle but it's something that the business world only pays lip service to and the these ideas that came into the 1940s from Hayek and free markets but when it comes to freedom within the companies that they

00:36:16

run most businesses operate in a paradigm that's a good 50 years older mired in the ideology of this guy Frederick Winslow Taylor and his idea of scientific management the idea that workers are just a particularly

00:36:29

unreliable kind of machine whose movements and actions should be scripted and constrained by an all-knowing management consultant who would work with an equally wise company series of company bosses to determine the one true

00:36:41

way that you should do your job in other words the ideology that led Toyota Kreme Detroit's Big Three automakers during the car wars in the 1980s so letting enterprise users do stuff

00:36:54

that they think will allow them to get there to make more money for their companies will sometimes make their companies more money also the scientific management is about as scientific as

00:37:04

trepanation and myers-briggs tests the business case for user rights is a good one but I really just wanted to get it out of the way so that I could get down to the real meat here the human rights case now it may seem a little weird on

00:37:20

its face to talk about human rights here but bear with me early this year I saw a talk by this guy Hugh hare the director of the biomech at mekin atronics group at the MIT Media Lab basically the

00:37:32

prostheses lab now some of you probably seen hairdo talks there's some amazing ones on YouTube it's electrifying now he starts out with a bunch of slides of all these cool prostheses that they've built in their labs there's you know legs and

00:37:44

feet and hands and arms and then there's even this device that uses focused magnetism to suppress activity in the brains of people with severe untreatable depression - amazing effect and then he shows this slide of him climbing a

00:37:57

mountain and he's buffed he's clinging to the rock like a gecko and oh yeah he doesn't have any legs he just has these cool mountain climbing prostheses and he looks at the audience for where he's been standing and kind of

00:38:10

pacing back and forth on the stage and he says oh yeah didn't I mention I don't have any legs and he rolls up his pants legs to show off these amazing robotic gams and he proceeds to run up and down the stage leaping around like a mountain

00:38:22

goat now the first question when I saw him that anyone asked was how much do those legs cost and he named a sum that would buy you a brownstone in central Manhattan or a terrorist Victorian in zone one London and the second question

00:38:37

that someone asked me was who's gonna be able to afford these things and he said well everyone if you're choosing between a 40-year mortgage on a house and having legs you're gonna choose legs which is

00:38:50

all by way of asking you to consider the possibility that there going to be people potentially a lot of people who are going to be users of computers that they don't own where those computers are

00:39:02

going to be parts of their bodies now I think that most of the tech world can understand why as you as the owner of your cochlear implants should be legally allowed to choose the firmware that runs on them after all when you own a device

00:39:14

that is surgically implanted in your skull it makes a lot of sense that you should have the freedom to change software vendors maybe the company that major implant has the very best signal processing algorithm right now but what

00:39:27

if a competitor patents a superior algorithm next year should you be doomed to having inferior hearing for the rest of your life or the life of the patent this is a problem that can't be overcome merely by escrowing the code the sort of

00:39:40

thing that you might do if you wanted to be protected against the company going bankrupt or by publishing the code the sort of thing you might do if you wanted to be sure there were no shenanigans going on this is a problem that can only be overcome by the unambiguous right to

00:39:53

change the software even if the company that major implants is a going concern and doesn't want you to so that helps owners but what about users now consider some of the following scenarios say

00:40:05

you're a minor child and you're deeply religious parents pay for your cochlear implants and they ask for the software that makes it impossible for you to hear blasphemy or you're broke and a commercial company wants to sell you ad

00:40:18

supported implants that listen in on your conversations and insert content jewelle adds that trigger discussions about the brands you love or your government is willing to install

00:40:30

cochlear implants but they'll archive everything you hear and review it without your knowledge or consent that may sound far-fetched but consider that just a few months ago the Canadian government was forced to abandon plans

00:40:42

to put hidden microphones throughout the nation's airports so they could listen in on every conversation going on and find the bad ones will the Iranian government or the Chinese government take advantage of this if they get the

00:40:55

chance and speaking of Iran and China it's these there are plenty of human rights activists who believe that boot locking is the start of a human rights disaster in countries like them it's no secret

00:41:06

that there are plenty of high tech companies that have been happy to build these lawful interception backdoors into their equipment to allow for warrantless secret access to communications these factors are now standard so even if your

00:41:18

country doesn't want the capability it's still there now in Greece there wasn't any legal requirement for lawful intercept on their telecoms equipment but during the 2004 and 5 Olympics bidding process an unknown person or

00:41:31

agency switched on the dormant capability and harvest an unknown quantity of private communications from the highest levels and then switched it off again no one knows who no one knows what they got no one knows what they did

00:41:43

with it or if they do they're not saying surveillance in the middle of the network is nowhere near as interesting as surveillance at the edge if you can control the devices instead of the network you can find out all kinds of

00:41:54

juicy things as the ghosts of misters Hayek and Marx will tell you there's a lot of stuff going on at the coalface that never makes it back to the central office and even so-called democratic

00:42:06

governments know this that's why the Bavarian government was illegally installing this Boone destroy honor a state Trojan on people's computers gaining access to their files and keystrokes and much else besides it is a

00:42:18

safe bet that the totalitarian governments of the world will happily take advantage of boot locking and move the surveillance right into the box you may not import a computer into Iran unless you limit its trust model so that

00:42:31

it only boots up operating systems with the lawful intercept or backdoors built right into the now assume we get an owner controls model we're in the first person to use a machine gets to initialize the list of trusted keys and then lock it with a

00:42:44

secret or another authorization token all this means is that the state customs authority has to initialize every machine as it enters the country before it passes into users hands now maybe you'll be able to do something to

00:42:57

override that trust model but by design the system will be heavily tamper-evident meaning that a secret policeman or a garden-variety snitch will be able to tell at a glance whether you've modified your computer to lock

00:43:08

the state out of it and it's not just repressive states of course remember that there are four major customers for sense aware spyware and lock where there's repressive governments large corporations schools and helicopter

00:43:23

parents that is to say that the needs of paranoid parents school systems and enterprises converge with those of the governments of Syria and China they don't share ideological ends but they

00:43:36

have awfully similar technological means to attain their individual ends and we're very forgiving of institutions as they pursue those ends you can do almost anything if you're doing it to protect

00:43:47

shareholders or children for example you may remember that there was widespread indignation from all sides when it was revealed that some companies were requiring worse prospective employees to hand over their facebook login

00:44:00

credentials as a condition of employment now these employers argued that they needed to review the lists of friends and what you said when you were in your private moments before determining whether you were suitable for working

00:44:13

their Facebook logins were fast becoming the workplace urine testing of the 21st century a means of ensuring that your private life didn't have any unsavory secrets lurking in it secrets that might

00:44:25

compromise your working life now the nation wasn't buying this from Senate hearings two popular editorials the country rose up against this practice but no one seems to mind that employers

00:44:36

routinely insert their own intermediate keys and to their employees devices their phones their tablets and their computers these allowed them to spy on their internet traffic even when it's secured with a little log showing in the browser

00:44:50

it gives the employers access to any sensitive site you visit on the job from your unions message board to your bank to Gmail to your HMO or your doctor's private patient messaging area to

00:45:02

Facebook there's a wide consensus that this is okay because the laptop phone or tablet that your employer issues to you is not your property their company property and yet the reason that

00:45:15

employers give us these mobile devices is because there's no longer any meaningful distinction between work and home corporate sociologists who study the way that we use our devices time and again find that employees are not capable of

00:45:28

maintaining strict divisions between their work and personal accounts and devices america is the land of the fifty five hour work week a country where very few professionals take any meaningful

00:45:39

vacation and when they do they go abroad with their BlackBerry's even in traditional old pre-digital workplaces we recognize the human rights of workers on company property we didn't put

00:45:52

cameras in the toilets took her tail employee theft by and large if your spouse came by the office while you were on your lunch break and the two of you went into the parking lot so that she or he could tell you that the doctor says that the cancer is terminal

00:46:05

you'd be aghast and furious to know that that conversation had been recorded by hidden camera and microphone but if you use your company laptop to access Facebook on your lunch break wherein your spouse conveys to you the

00:46:17

fact that the cancer is terminal you're supposed to be okay with your employer knowing that because they've been running a man-in-the-middle attack on your personal life and on the most intimate into intimate details thereof

00:46:31

there are plenty of instances in which rich and powerful people and not just corporate peons or children or prisoners will be users instead of owners of the devices around them every car rental

00:46:43

agency would be love would love to be able to not just LoJack the cars they rent you remember cars or computers you put your body into they'd love to log all the places you've been for marketing purposes and analytics there's lots of

00:46:56

money to be made in finagling the firmware on your rental car GPS to ensure that your routes always take you past certain billboards but in general the poorer and younger you are the more likely you are to be a

00:47:07

tenant farmer and some feudal lords computational lands the more likely that your legs will cease to walk if you get behind on payments meaning that any thug who buys your debts from a payday lender

00:47:20

can literally and legally threaten to take your legs or eyes or ears or arms or insulin or pacemaker away if you don't come up with the next payment now before I discussed how an owner override

00:47:33

would work some combination of physical physical access control tamper evidence designed to give users of computers owners of computers rather the power to know and control what bootloader and operating system are running on them now

00:47:45

how would he use your override work I think effective user override would have to leave the underlying computer intact as a first design principle now here's a totally hypothetical model for this this

00:47:58

is the bit that I'm telling that I mentioned before it's not a technological proposal it's an example imagine that there's a bootloader that can reliably and accurately report on the kernels and operating systems it

00:48:10

finds on the drive this is the prerequisite for any of these scenarios the one in which the state of the corporation gets to control how owners use of the devices the one in which owners get to use their devices and the one in which I'm about to describe and

00:48:22

which users get to control their devices now give this hypothetical bootloader the power to suspend any running operating system to disk encrypting all of its threads and files so that the person sitting at the console can no

00:48:34

longer access them so you walk into an internet cafe or some other context in which you're using a device that you don't own and you park the operating system and you now have the power to

00:48:47

select the OS from another from a thumb drive or from the network and now imagine that the internet cafe has a some kind of lawful interception

00:48:58

backdoor for the police now you you're an attorney you're a doctor you're a corporate executive or just a human being who doesn't like the idea of your private stuff being available to anyone who's friends with a dirty cop so you do this three finger salute with your F

00:49:12

keys to drop into this minimal bootloader shell one that invites you to give the net address some alternative OS or insert your thumb drive the cafe owners operating system is parked you can't see it anymore but the bootloader can assure you that it's

00:49:24

dormant and not spying on you as your OS fires up and when your OS has done all of its working files are trashed and the bootloader confirms it not just because this keeps the computers owner from spying on you but

00:49:37

because it stops you from attacking the users owner or other users that come by there will be a technological means of server ting this but there's a world of difference between starting from the design spec that aims to protect users

00:49:49

from owners and vice versa than one that says that users must always be vulnerable to the owner's wishes fundamentally this amounts to the difference between freedom and openness between what we've called free software

00:50:02

and open source for all these years for a long time these have seemed like distinctions without a difference I think that we are slowly acquiring the meaningful difference between them now human rights and property rights often come into conflict with conflict with

00:50:15

one another for example landlords aren't allowed to enter your house without adequate notice even if you rent it from them in many places your hotel can't throw you out if you're paying the rate even if you overstay your reservation the repo man can't come and take your

00:50:28

car away without going through some procedure and giving you the opportunity to rebut an accusation of being delinquent in your payments and when these when these processes are streamlined to make them easier for the

00:50:40

property owners we often see human rights abuses like when Robo signers working for eviction mills use fraudulent declarations to evict homeowners who are caught up on their mortgages or even people who don't have mortgages the potential for abuse in a

00:50:54

world made of computers is much greater imagine your car driving itself into the repo yard or your high-rise apartment building switching off its elevators in climate systems stranding thousands of

00:51:06

people until a distributed lot until a disputed license payment is made now this already actually happened once before back in 2006 there was a 314 car

00:51:17

robotic parking model RPS 1000 garage in Hoboken New Jersey that took all the cars and its guts hostage it locked down the software until the owners of the garage paid a licensing bill that they disputed

00:51:30

they paid it even though they maintained that they didn't owe it because what the hell else were they gonna do there were 300 cars trapped in the belly of this machine what will you do when your dispute with a vendor means that you go blind or deaf or lose the ability to

00:51:43

walk or become suicidally depressed the negotiating leverage that accrues to owners / users is total and terrifying users will be strongly incentivized to settle quickly rather than face the

00:51:55

dreadful penalties that could be visited on them in the event of a dispute and when the owner of the device is the state or a state size corporate actor the potential for human abuse rights skyrockets this is not to say that owner

00:52:09

override is an unmitigated evil think for a minute of these smart meters they can override your thermostat at peak levels without the ability to tell your house to change its HVAC at the right moment we won't be able to switch off coal and other dirty power sources that

00:52:23

are the sorts of things that we can vary and ramp up at peak demand but these things work best if the users the homeowners who allow the party company the power company to install them in their homes can't override them but what

00:52:36

happens when griefers or crooks or a government trying to quell a popular rebellion use this to turn off the heat during 100-year storm or crank the heat up to maximum during a heatwave the HVAC in your house can hold the power of life

00:52:49

and death over you do we really want it designed to allow remote parties to set policy on it that you can't override the question is wants me to create a design norm of devices that users can't

00:53:00

override how far will that creep especially risky would be the use of owner override to offer payday loan style services to vulnerable people if you can't afford artificial eyes for your kids will subsidize them only if

00:53:14

you let us redirect their focus to sponsor toys and sugar snacks at the store but foreclosing on all an owner over I probably means that there's going to be poor people who will not be offered some technology at all if I can

00:53:26

LoJack your legs I can lease them to you with the confidence of my power to repo them if you default on payments if I can't I may not lease you legs at all unless you're already rich but if you but if your legs can decide to walk to

00:53:38

the repo Depot without your consent you will be totally screwed the day that's some mugger or Pistor griefer or the secret police figure out how to hijack that facility it gets even more complicated of course

00:53:51

because you are the user of many systems in the most transitory way subway turnstiles elevators the blood pressure cuff at your doctor's office a public bus or an airplane it's going to be hard

00:54:03

to figure out how to create a user override that isn't nonsensical although we might start by saying users are someone who are the sole user of a device for a certain amount of time as I said at the start of this talk this is

00:54:15

not a problem I know how to solve unlike the coming war on general purpose computing the civil war over them seems to present a series of conundrum without a single clean line of solutions which is why I'm talking to this audience

00:54:27

about them these problems are way off but we're supposed to be thinking in the long term here and of course they'll only arise if we win the war on computers first but come victory day when we start planning the

00:54:39

constitutional Congress for the new world we're regulating computers is acknowledged as the wrong way to solve problems let's not paper over the division between human rights and property rights this is the sort of division that while

00:54:52

it festers puts the most vulnerable people in our society in harm's way agreeing to disagree on this one is not good enough we need to start thinking now about the principles that will apply when the day comes because if we don't

00:55:05

start now it may be too late thank you a certain an analogue question mm-hmm you've been in Britain how long often on since 2003 they've been doing more

00:55:27

surveillance cameras than almost anybody yeah there must be some serious parallels between what's been going on what's playing out there versus some of these human property privacy issues you're raising in terms of computers

00:55:39

what have you got so far from that what have they got from that well you know I think one of the big problems with the UK is the conflation of solving crimes and preventing crimes it seems like there's a lot of times where cameras

00:55:51

solve crimes but they often seem to serve as a as a substitute for for the kind of boots on the street that might prevent crime so for example I know someone who was a murdered near his house he came up out of a tube station

00:56:04

there used to be a human guard there and that human guard had been replaced by a camera and that camera did capture the three kids who followed him out of the tube and stabbed him to death and took his phone and they were later captured on the basis of it but he's still dead

00:56:15

right I think that that cameras aren't the same thing as people and moreover I think that cameras kind of change our behavior in in lots of ways that we're still making sense of and they lead us

00:56:30

to weird and nonsensical conclusions I mean Britain has this love-hate relationship with cameras they will arrest you for taking out your camera but they will take your picture over and over again I the weirdest of these is the daycare that we take my daughter to

00:56:42

every day we walk 10 minutes to to the to the daycare and you know Britain has like 14 CCTVs per red blood cell now which means that we're photographed like a million times on the way to the daycare and when you get to the daycare

00:56:55

there's a CCTV over the door and beneath it is a sign that says in the interest of safety please don't turn on your camera in the daycare which is to say we can take lots of pictures of you but you can't take any pictures of us I was IRA

00:57:08

prized my role this year as the atheist Jewish Santa Claus of the East End day care and we weren't allowed to take any pictures of any of the of me because there might be a kid in it and once you

00:57:20

take a picture of a kid I don't know what you'll do with it but somehow it's linked to pedophilia god I've never understood it myself which somehow erases all argument for

00:57:33

some so it's been going on that long in Britain or this is the populist sort of moving in some direction relation to all that as the government moving in some relation to all of that or is it they

00:57:47

put up all the cameras and that's I think the fraud has been boiled I mean nobody can really articulate what it's supposed to do and and it is one of these things there's a whole class of you know old lady who swallowed a fly problems we're having swallowed the fly

00:57:59

you need to swallow a spider and a bird and so on where you know if the CCTVs aren't solving the problem the problem must be that there aren't enough CCTVs and you can like like any problem can be solved by just adding more CCTVs and

00:58:12

they oftentimes you know one of the problems with these things is that they well they're a suite of them one is that weirdos and bad guys might be looking out of them right so that puts you at risk one is that face that when you have a lot of them it's very hard to keep

00:58:25

them all working so the important you've got this sort of maintenance rota that doesn't distinguish between CCTVs that are in places where you know the research shows you that you can prevent crime like parking garages which is places where people are often on their

00:58:37

own and where it's you know quite easy to pull off a crime and get away clean and the CCTV that's just sort of reflexively puts somewhere because CCTVs add magic security dust to any public

00:58:48

location and and as a result you you end up with a kind of because there's no theory behind it there's no one's out an articulate able theory about safety that relates to CCTVs except CCTVs make us safer

00:59:00

there's no way to deploy the system rationally it's just it's just doctrinaire question from poll clip oh it's interesting he's got two questions only the first one is here given the

00:59:14

move to cloud computing don't we all become users that's certainly true although you know you could imagine like some combination of cryptography and virtual machines making you the owner of

00:59:28

our owner and some meaningful sense of a computer that you're not physically present at but yeah I mean there's certainly lots and lots of computers that we use every day that we don't own

00:59:41

and one of the things I'm getting from all right there's a really good question here mmm

00:59:51

basically the question relates to this has been going on for a while what you're talking about and what you're saying is it's going to keep going on for a while so the 50

01:00:06

years plus or minus of dealing with computers that we're in that are in us that we live in that have given us a set of trends that you're responding to and

01:00:19

that as a science fiction writer you're then thinking where those trends go and what makes them flip into something else and so on what is your overall sense of the last 50 years leading to the next period of time well so I think for the

01:00:33

last maybe for the last 20 years anyways that kind of the period of the drm Wars okay there's been the sense that property owners rights are being taken away there's this fight between like a

01:00:45

kind of notional property and real property that that because everything that you own contains something licensed that you are you're that kind of clean ownership that's in some ways illusory anyways but that clean kind of blackwell

01:00:59

Ian's ownership Blackstone Ian's ownership where you have the you know absolute Tyranny over your device is being eroded that that you know that your your real property rights in your TV or your phone or your computer are

01:01:12

being taken away because they're being governed by EULA's that kind of you you you agree to just by sort of by standing here and shouting no no no I don't agree you agree that I'm allowed to come over to your house and clean out your fridge and wear your underwear and make some

01:01:24

long-distance calls you know so I think that that that has been a building consensus right that there's a lot of people who really agree with that and and I I agree with it that and that this

01:01:37

is improper that that it's that whatever it is that we use to kind of regulate information it shouldn't be at the expense of the right to you know take a picture of your shoes even though they

01:01:50

embody a design path owned by someone or take a picture of your t-shirt even though it has some copyrighted mark on it or you know record your kid dancing in your kitchen

01:02:03

and put put the adorable toddler on YouTube even though there are some faint strains of prints playing in the background right these these things come up and they are they're important issues and I think that by and large we go ya

01:02:16

know that's right we should you own it you bought it you own it it's yours whatever it is we do to regulate information let's not do it at the expense of of owning stuff but there's also this increased consensus I think

01:02:28

that like your company's IT department has the right to tell you what you do with your computers and if you don't like it you should work at a different company and I think that that's that's

01:02:41

our reflexive answer that's not a very good answer you know I wrote a column about the fact that we weren't pissed off when when Facebook logins were captured by employers when users used

01:02:55

their company computers to access Facebook even on their lunch breaks or whatever but we were pissed off when when prospective employees were required to give their credentials the idea was like and the email that I got from this

01:03:07

universally said you're an idiot right IT departments have the responsibility and duty first of all to spy and everything their users do because how else would you stop company secrets from

01:03:19

leaking out or house would you have HIPAA compliance or how else would you have some other compliance regime in the enterprise and then they also said you're an idiot because it's your it's your boss's computer why are you doing

01:03:31

something personal on it it's it's you know if you want to do something personal use your own computer but you know we're not we're not really good at maintaining clean divisions between user computers and owner computers and and moreover as I said in the talk you know

01:03:44

there's lots of times where you're on your employer's property where your employer's dominion over your inter relationships with other people and you're kind of your person is not

01:03:55

opposite absolute so why should it be absolute just because there's a computer in the middle of it why is it that Facebook credentials are beyond the pale for prospective employees but you know justjust just a thing that you

01:04:10

had you should sort of suck it up and if you don't like it use someone else's computer once you take the job I guess since you're saying that this that you're talking about property rights and human rights and the property rights

01:04:22

seems to be an endless condition of negotiation and battle and a civil war and so on whereas the human rights I get the sense you're saying is a little more absolute and it doesn't change so much

01:04:33

over time yeah yeah yeah I mean well well Human Rights always then be able to sort of clarify the issues no I actually I don't think human rights are unchangeable over time because I think

01:04:47

that there are human rights questions that we've never had to answer like control over your sensorium or universal surveillance or even negotiating the difference between you know like we've

01:04:59

always had you know a certain abhorrence for people who go and read your private diary in order to incriminate you I'd have me once but but horrible but but I mean what about your life logger what about your Fitbit what about you know

01:05:12

there's a kind of depth of information captured there that on the one hand seems kind of like impersonal right and yes there you go you know like like your

01:05:26

pulse at any given time you know why why should that be private but at the same time could be quite damning taken as a whole I kind of feel like these disclosures they're a bit like involuntary disclosures or even

01:05:39

voluntary ones sometimes are a bit like smoking right taking a puff of a cigarette won't give you cancer but if enough people take enough puffs of cigarettes some of them will get cancer and being disclosing once won't put you

01:05:53

in harm's way probably but enough disclosures over wide enough population is going to create some really weird pathological outcomes and when those disclosures are compelled because they come out of the devices that is even

01:06:05

more potentially harmful so as part of this driven by just the pace of change because norms emerge laws emerge regulations that people feel okay about emerge

01:06:18

but if you got Moore's Law chugging along behind all of this and just making it a whole new engineering and you swirl every few years does that mean that norms never get to catch up and there's

01:06:30

always gonna be Civil War for the foreseeable future well I mean I don't think that it's that it's merely future shock or you know acceleration shock I think that like you know people I sometimes hear people say privacy is

01:06:43

dead but it seems to me that privacy is dead is not a description of the world as it is but rather a slogan right I need more money privacy is dead okay no it's it's it's it seems like whenever

01:06:55

I hear someone declaring privacy dead that person has an interest in privacy being dead and and you know they they're they're it's it's not a neutral it's not

01:07:06

a neutral observation and so you know Larry Lessig talks about these four regulatory forces code law technology and norms and I think that when new

01:07:19

technology comes along there are lots of norms differing norms that emerge but the ones that emerge victorius are often those that are prompted by people or

01:07:31

supported by people or support the interests of people who already have lots of power so you know for example I can't figure out how to rent you music and then rent it to you again later

01:07:45

unless I can control your computer therefore controlling your computer is bad when you know the another answer to that might be why is renting your computer and absolutely why is renting music to me an absolute right I mean if

01:07:57

you can't figure out how to rent computer music to me without doing something so obviously wrong then you should get another another line of work right it seems to me that like there's nothing a priori like if you are a

01:08:10

Martian watching the earth through a telescope there'd be nothing about one or the other that would tell you that that seems like on its face to be a more reasonable claim right you know renting music not renting music there's there's

01:08:22

no there's no absolute kind of necessity that music be rentable question for him John Foster another question he says thank you for validating my vintage

01:08:33

computer fetish old hardware makes it all future-proof yeah well no I mean only if you don't have a modern car or only if you don't get fitted with a

01:08:46

pacemaker only if you don't walk past CCTV use only if I mean there are lots of computers that you will end up using that aren't visibly computers right nothing is future-proof yeah well I don't I mean I think being a hermit

01:08:58

maybe but but you know that I you know there's that is a very you know you versatile quote just because you're not interested in technology doesn't mean technology isn't interested in you you

01:09:09

know a little little warmed-over trotsky their red diaper right yeah a little

01:09:29

fanboy question you're about to finish a book with Charles Stross no no no no it's been long garnished just long finish yeah yeah no it's say something out in September so soulless as well you may have read the first two-thirds of it

01:09:42

we with the first story we ever wrote together it's called jury service and it's a comic novel set in a world in which all of the technophiles have ascended to the singularity they're all there at they've all abandoned their

01:09:55

bodies and and left behind all of the Luddites the all of the people who are deeply religious and reject technology and so on so the it's a it's the inverse of the Left Behind novels in which all

01:10:08

of the pious people go to heaven and this one all about all of the refuseniks are left behind and all of they all other kind of highly secular technophiles go to the cloud and it's

01:10:20

it's and the cloud sends spam to the earth it has it has ideas in fact it's this it's disassemble the whole solar system except for the earth because it has a sentimental attachment to it and it's

01:10:33

fun this giant Dyson Sphere and there's this just this one lighthouse beam tracking the earth as the earth goes around the Sun and the rest of the sun's radiant energy is being absorbed by the bones of all the planets that are now you know grinding computronium running

01:10:46

simulations of nerds and and the Nerds send send spam to the earth with like cool ideas they've had for what me people can do and and there's there's a jury that meets periodically to figure

01:10:59

out which of these technologies are useable so that the protagonist of the story is a bit like rincewind and the terry pratchet almost kind of comic hero is summoned to libya to attend a

01:11:11

people's technology court to determine whether or not this technology is useable so we wrote that then we wrote a sequel called appeals court so two novellas so toda books really liked it and said write a third we'd always talked about writing a third so we wrote

01:11:24

a third which is called parole board and then we rewrote all three to make them all fit together as a single unitary story and that's that's called rapture of the Nerds and that's coming out in September and we'll be touring with it

01:11:37

if just just a little baby tour Charlie's gonna be over for the world science fiction convention in Chicago and I'm gonna be at Burning Man and we're gonna rendezvous I think we start in New York and we're going to like New York Lexington and Boston and then

01:11:51

Rochester there's the thing at RIT and then we're going home so yeah that's that's the that's the next book okay so one of things you're great at and keeps me reading your books as you you make

01:12:03

this wonder seeming thing come to happen the gold farmers get together and they organize whatever it may be and here you've got the rapture the Nerds in their Dyson Sphere making all this cool stuff happens so they've attained their

01:12:15

rapture but the world doesn't stop what happens then well I mean it's a it's a series of comic stories that are in part about the arrogance of technological determinism

01:12:29

of telling people well this is this is just gonna happen get used to it here's the technology change things so in some in some sense a kind of self critique because we're as guilty of that as anyone else and it's also you know in the third and the third act it's really a book about

01:12:42

how the fact that we can all ascend and and do things in computers that are impossible in the real world without the constraints of the real world doesn't mean that they'll be very nice and in

01:12:56

particular doesn't mean that they'll that they'll be very aesthetically pleasing so the the third act is really about life without constraint inside a

01:13:08

simulated universe in which everything kind of looks like Geocities page and and there's a certain that that is that is not necessarily a prediction as much as an observation of the world that we

01:13:22

inhabit today you know and and that's it's it's so we open that third act with someone pounding at the door and the person who lives at the house says who is it and the voice says it's the

01:13:35

singularity and the person says we don't want any and the voice at the door pounds some more and says everything is different now I don't want any and and so it's a it's a bit of this kind of

01:13:48

idea that there that there may be legitimate reasons not to want any so you favor a certain variety going forward it sounds like well I think that

01:14:01

you know so I had I was hanging out with Dan Gilmore the other day that no journalist and he he talked about this thing he used to do where he maintained a list of 10 things that he believed were true and the first one is like

01:14:12

Microsoft is a dirty monopolist that needs super adult supervision and every six months he would call up people who totally disagreed with him about the ten things that he thought were true and get

01:14:25

them to tell them their best arguments to figure out whether or not he still thought that they were true as a means of kind of sharpening his belief and understanding whether or not he was taking things for granted as he said you know if you believe something to be

01:14:38

absolutely true in an era of rapid technological change and you never revisit that belief the one thing you can be sure of is that you'll be wrong eventually and so I kind of feel like we need to investigate the possibility that we're not right and that's one of the

01:14:50

things that happens in this book plus it was really fun it was really really fun to do this I I did a joint talk I don't know if it was ever put online with a guy named Peter Biddle who's a really good guy now works

01:15:01

for Intel but he designed palladium he designed trusted computing for Microsoft and he's totally on the other side of a lot of these issues for me then from me and we gave we gave a talk where we reverse positions where he gave the

01:15:14

position against it and I gave the position for it and that was really good fun trying to come up with the best arguments the arguments that I that I always hope no one would ever bust out when I was arguing about it because I

01:15:26

couldn't think of the good one so um arguing with yourself what have you changed your mind about lately well I

01:15:38

mean TPMS I've been really thinking a lot lately you know as I as I contemplate sensor density in the devices that I own I get I'm getting increasingly worried that I need a means

01:15:51

of verifying what's running on them I you know and and lots of things are making me paranoid about that things like the bone destroy honor the the fact that it seems like there's a migration path from what we call like advanced

01:16:05

persistent threats there are the kinds of things that governments do into automated attacks where it's not it's nothing personal it's just a thing that you can kind of execute against a population of 10 million people until

01:16:17

you find the person who's vulnerable I just saw a talk at Def Con kids actually by Moxie Marlinspike and a couple of friends over vulnerabilities in the near field communication sensors and Android

01:16:28

phones and he took a piece of what looked like paper and he brushed it against a phone and thereafter the phone belonged to him and he said well there's some limitations you need to get the paper very close to the phone the phone

01:16:40

needs to be switched on when it's running but I'm like what if this was what if this was confetti on as they dropped the ball on New Year's Eve and everyone had their camera phone out without working he was like yeah that'd probably work cuz thereafter you own the

01:16:53

phone over Bluetooth or over the public Internet you don't have to be four inches away to control the phone you just need to be four inches away to put the phone in a state where you can control it and I kind of feel like knowing what's going on in my computer

01:17:05

is becoming increasingly important i increasing ly find myself putting post-it notes over my webcam you know being a little worried and I didn't used to worry much about that

01:17:16

there's paranoia and so there's paranoia as a sport right lots of people enjoy I'm gonna be totally secret and it's it's hard and it's interesting and so on

01:17:31

it is yeah it's a fun game right it's a little it's a great mind game to play is it worth it well so I think that it's so the paranoia here is not the fear that

01:17:43

someone is out to get me I don't flatter myself that anyone's particularly out to get me the fear is that some vast automated attack against a range of ten million IP addresses

01:17:55

might catch me in it and that that might compromise me either in small ways or in large ways and that that compromise could come now or further down the line and the same way shrugging we boutta know what's the problem well so you

01:18:07

remember we all thought that Usenet was a book written on water and we wrote things on that that we didn't intend to ever have repeated does anybody died because of something they wrote on

01:18:20

Usenet back when no no absolutely not but but for example I think people in in the Middle East have died because of things that were from their computer that's free yeah no I think that's that's absolutely true

01:18:33

and and the thing about Usenet was there came a day when it turned out that there was an archive of Usenet Dejan News switched on and then Google bought that archive and Google seems to have Aden oh shut that archive down I can't seem to find it anymore but for a while you

01:18:45

could find kind of you know high-flying calm CEOs five years earlier talking about being in the K hole at Woodstock and you know this was not the kind of thing you wanted on you know the eve of

01:18:57

your IPO and it was it was it was a sermon that he died come on nobody died but it was a big man right and I think that like we are starting to see for example the the first glimmering of what

01:19:11

happens when facial recognition is applied to the corpus of all photos ever put on the internet and we're not far away from every photo on the Internet being labeled automatically or some fraction

01:19:22

of every foot on the internet being labeled automatically and and and then there being you know social graph analysis against that of people that you never wanted to disclose you knew and that might lead to people dying or

01:19:34

losing their jobs or later being compromised in some way small or large and I think that there's that so I don't worry particularly that someone will attack me I worry that if something gets extracted from my

01:19:47

computer that later on it might be correlated with lots of other things and that I might come to regret it later I think we're really bad at judging the net present value of a future option on a disclosure right we don't know what

01:20:00

it's going to be worth in 20 years is why Facebook is such a problem because if the basis of Facebook is people can can rationally measure the value of a of a disclosure today again against the

01:20:12

value they get from Facebook and and the total value of that disclosure over the life of that disclosure living on Facebook the empirical evidence is that they're really bad at that at that valuation it's like some of what we're

01:20:27

talking about here is the difference between anticipatory problem solving and retroactive problem solving and this is Jimmy Wales parable the steak knives yeah sure you know where is programmers

01:20:38

wanted to say well our users might do this bad thing here will write this really nifty software prevent them from doing that and Jimmy said wait a goddamn minute see if there's actually a problem and then solve that much easier to solve

01:20:50

a problem you can see then a problem that you can imagine because the problems you can imagine are infinite well we already have lots of weird problems where people take over computers then that that they people who own them or use them I mean Lower Merion

01:21:03

School District was the canary in the coal mine here affluent suburb of Philly they issue every student a MacBook the students are required to use the MacBook to do their homework they're required to bring it to school every day and home

01:21:14

every night and what the what the school district did was they installed covert software that allowed them to operate the webcam without turning on the little green light and in theory this was to stop thieves but in practice it turned out it did they were just watching the

01:21:28

kids lots of times and the way that they discovered this is there was a kid who'd had a long run dispute with this principal he was a discipline case principal called him in and says I got you now sunny you've been taking drugs hands him a photo of the kid that were taking pills in his room

01:21:40

and the kid says first of all that's a Mike and Ikes candy and second of all how'd you get a picture of me in my bedroom and it turns out that they'd taken thousands of pictures of this kid at home and at school naked and dress

01:21:52

with his parents on his own and you know that's that's an inkling of what can come because it's durable that what happened was it corrected or on to this day what they were sued right they were sued they lost and they got into trouble

01:22:05

that in other schools that version well no in fact I was just at DEFCON kids and I gave and talked and I and and I gave a talk about school networks and school computers and the control they exert

01:22:17

over their kids and the kids put his hand up and he said oh my computer does way more than this kind of network censors censorship and surveillance stuff my computer every 15 minutes puts up a dialogue reminding me that they can

01:22:30

watch me and see me and see everything I do and and and so they're just not keeping it a secret anymore right no software but are you being fearful enough by entering by entering this

01:22:42

space you agree that you will be recorded right you've seen those signs in fact I just landed at some airports somewhere that said and like on the in the customs cue we are video recording today if you object to being recorded

01:22:55

please don't enter this area the customs cue right you are being video recorded through your school computer if you object to being recorded drop out of school I

01:23:07

don't know be a be a truant well there's a guy here in town will give you money to drop out of school after my own heart Peter Teal'c is serious money the people draw from Burning Man so you went to

01:23:22

Burning Man last year I gather and you're gonna skip the science fiction convention this year to go to Burning Man again what's that about yeah I'm skipping the world science fiction convention is here for the first time the second world kind of mist in about 18 years 19 years now Hiro's going to

01:23:36

Burning Man yeah mm-hmm though here it's going to be in my camp at Burning Man I think some of my camp mates are here hi guys what's your camp it's where I'm camping with the liminal labs who are these awesome folks I know from when I

01:23:49

lived in San Francisco and when in the heyday of the well and they have this amazingly organized camp that we camped in last year for the first time it was my first time at Burning Man and I'd

01:24:00

always assumed that Burning Man would be two things it would be very austere because who the hell can be bothered to bring stuff out to the desert and and it turns out that that's totally wrong but it's incredibly lavish that everyone

01:24:12

takes it as a challenge let's see what kind of crazy stunt I can pull off in the desert and I also thought it'd be very doctrinaire I thought that it would all it would always be about the that you know these ten principles and then everywhere you went there'd be people quoting the ten principles like mouths

01:24:25

little red book at you and in fact although the ten principles are clearly central to what you do at Burning Man and anytime anyone quotes them they do so with an ironic I roll knowing how weird and ridiculous it is to be quoting

01:24:37

these ten principles can you state some of the gen principles and doctrine EU enough to gosh oh and I remember there's no dogs I participate don't SPECT it I probably can't get no spectators Leave

01:24:50

No Trace or some other you Leave No Trace what else gift economy what immediacy imagine see so radical self-reliance remember the ten rules

01:25:05

and so it was it was so transformative and amazing that I'm going back this year and skipping the world con and then for the next like five years it looks like the world com won't overlap world con so I won't have to one overlap Burning Man so I won't

01:25:17

have to choose so I'll get to do both so gift economy you know a red diaper baby comes along and here's 50,000 people doing no money and they're doing a gift economy what does that look like um I guess the thing that I had a hard time

01:25:30

wrapping my head around was the difference between gifts and barter and I think everyone has that when you talk about gift economies two people have never been they say well what do you give people and what do they give you in exchange and this seriously

01:25:41

misunderstands the nature of a gift economy and it's funny cuz it's infected the way I think about child-rearing so one of the things that I often get for my daughter is a kind of horse trading she's old she's four and a half you know

01:25:53

and and it probably comes from us because I'm sure we end up doing it because everybody ends up doing it if I give you an Apple will you put your shoes on so we can go to daycare but that that always as you know every bad

01:26:05

habit you have comes back into your kids mouths so I'll put on my shoes if you'll give me an apple and the thing that I keep trying to come back to with her is I will give you an apple because I'm your father and I love you and we live

01:26:17

in a family together and you get apples you will put your shoes on because we're in a family and your job is put your shoes on these two acts happen nearly simultaneous but they are not causal

01:26:28

right and it's a it's a difficult it's a difficult distinction to make but it is a distinction with a difference right I mean you know this is this whole notion of pocket money is something as

01:26:44

something that's not related to chores chores or chores you do the chores because you're part of the family we all do what we can you get your pocket money because you're part of the family who all do what we can if I if people stop buying my books and my wife was supporting me I wouldn't ask

01:26:58

her for a permission to use the family checking account because I'm part of the family unit and we used the checking account together for the same total you stay married one checking account yes oh well

01:27:11

and by the same token my daughter when she's old enough will get her pocket money because she gets pocket money and she'll do chores because she does chores and I hope that there will be some way that we can keep those things separate

01:27:26

by Mike there's something as you know saw its law here might the technology went wrong they changed my mic and then I keep knocking this one off after having this other one taped on so so how old does your daughter need to be pretty take her to Burning Man I think that's

01:27:40

more question about us than her it's it's really a question about the extent to which would you take her now if you lived in California what you might I gather yeah I mean it's possible someday we might end up living in California we certainly talk about her lot we've we've

01:27:52

lived here before and for a long time my wife and I lived here for fourteen months when when I had the Fulbright at USC and she was working for the BBC and certainly it something we think a brother here and yeah little brother and the sequel homeland which is

01:28:06

coming out next February both set in San Francisco a little bit cheek woman so homeland is the sequel and it's it's set a couple of years after little brother and it's it's set in a world in which he

01:28:20

has the young man who's the protagonist has remind us what the protagonist does he and little brother that it's it's about kids who are out playing a videogame or an alternate reality game in San Francisco one day when someone

01:28:33

blows up the Bay Bridge and the city goes into lockdown on the DHS is here and no one can figure out what's happened and someone arrests them and because they're weird and suspicious why are these school kids walking around

01:28:46

what's going on and instead of explaining what's gone on they say I'd like to see a lawyer and that turns out to be the wrong answer to give in a condition of emergency for some vote for some value of wrong answer for for what

01:28:59

certain people seem to think you should do in the case of an emergency and as a result they end up being treated very badly and this is the object lesson they need to tell them that they that that what they must do is declare war on the Department of Homeland Security and

01:29:12

restore the Bill of Rights to San Francisco and so that's what the rest of the book is about this this is a young adult book a young adult novel it's and the sequel to it is a book

01:29:21

about what happens when the emergency is not a visible one but a kind of frog boiling one when when some combination of economic collapse and austerity and

01:29:37

and and you know the slow monotonic changes instead of this fast kind of Shock Doctrine change ends up in this with getting to almost the same place and so it opens with with the young

01:29:51

protagonist having been unemployed and dropped at a university because he couldn't afford his father's lost his job at Berkeley after after contractions and the staffing there he no longer gets free tuition tuition is going up tuitions going up he can no longer

01:30:04

afford it he's especially conscious of the fact that his student debt will follow him on to his social security and and he walks away and and he's trying to find a job and he's pounding the pavement he can't find a job as parents can't find a job

01:30:15

everything's contracting he's he's he's hanging out at the hack space is hanging out at a noise bridge and he manages to thank you yeah and he manages to scrimp and save and and go to Burning Man where

01:30:26

he which incidentally makes my trip to Burning Man tax deductible he he finds a job he meets some people including

01:30:40

someone who knows some people who know a muckraking independent candidate for the California State Senate who's looking for a webmaster and he walks into a job but he walks into a job at the same time as he walks into holding a giant stash

01:30:53

of government leaks and so he it's about his tension balancing the fact that he's got a straight day job working for this muckraking senator and all of these leaks and without giving away the whole story one of the leaks is that an

01:31:05

organization a bit like Blackwater has spun out and has become private debt collectors who are buying up subprime debt and and securitizing it and collecting on it and they have fanned

01:31:18

out lobbyists to all 50 state houses in the country with a proposal to pass state laws on debt collection that will allow them to attach the assets of parents of students who've gone back home after graduating and not finding a

01:31:32

job so it's basically another way of taking away your house and it's you know that this is this is the secret he's sitting on at the same time as he's working for this muckraking

01:31:43

campaigner so it's a there's there's lots of occupy style stuff there's lots of UAVs and quad copters there's lots of WikiLeaks style stuff there's lots of there's lots of tour and forward secrecy Jake Applebaum we're at one of the

01:31:55

afterwards and Aaron Swartz wrote another is lots of sort of 21st century politics and and you know next generation political campaigning and the kind of stuff we did with SOPA and PIPA in there and and some ideas that Aaron

01:32:07

had Aaron actually basically wrote a passage of it for me about what the future of political organizing and campaigning might look like in a post sefa PIPA SOPA world so it's it's it's a lot like little brother only the next

01:32:19

thing and this is young adult too and it's young adult too Oceania adults are very grateful keep it up thank you thanks [Music]

01:32:34

[Music] you