Video - Hardware, Software, Trustware

In this talk, Andreas speaks about the ongoing clash between hardware and software development (instead of "East vs. West") within the emerging culture of trustware. He also added how importance these 3 components on a developing society.


The title of this talk is Hardware, Software, Trustware. How many of you have been paying attention to this whole business about a fork. Did anybody heard of a fork?

There is a culture gap that is being expressed in this fork and most people looked at this and they see the obvious culture gap. Most of the mining happens in China, most of the miners are Chinese. A lot of the development happens in western nations, a lot of the developers are from western nations and at a first glance it looks like the culture gap is some kind of East-West thing, it’s not. In fact I think a much better framework for thinking about this is that the culture gap at the center of the debate we’re having today is a culture gap between people who built hardware and people who built software and those cultures have been diverging since the 1950s.

If you’re involved in computer science, if you do hardware of software for a living you know what I’m talking about. Software goes way back but not as far back as hardware because the very first software was hardware and if you wanted to use a computer it ran one program. Some of the first computers run very specific programs probably one of the earliest examples is the Enigma-cracking machine at Bletchley Park built by Alan Turing to crack the German cryptography during World War II. It did not have software per se. It had inputs, you could input the message and it will try to figure out the key but it only ran one program and you could barely really call it a computer. It was an electromechanical device with a bit of electronics, very primitive.

Software started happening in the ‘60s and software represented a giant leap forward because until then if you wanted to reprogram a computer you had to change its wiring or you have to flip a lot of switches. Like a lot of switches, like 10,000 switches in a big bank of switches and if you got one of them wrong that was a bit of a problem. Programming in binary, not fun. And out of this gradually we started having these two cultures emerged. The culture of people who built hardware and the culture of people who build software and the fundamental difference has to do with the life cycle of development and that persist to this day.

When you build a hardware device your life cycle is measured in months if not years. Eighteen to twenty-four months. You design a chip to do something, you make some architectural decisions these decisions will arrive at the market place two years from now and if you got them wrong you start again. If you make a mistake in hardware things happen and you can’t issue a patch. If you ship a phone that has a nasty tendency of blowing up there is no software patch that you can issue that will fix the battery issues. It’s out there in people’s pockets getting hot.

The culture that comes out of that is a very conservative culture. The planning timelines for building hardware are very long and they require absolute precision. If you make a mistake you don’t just change your line, recompile, try again, issue a patch. You recall a billion dollars worth of silicon and turn them into scrap because you made a mistake, a nanometer scale mistake. And at first software was like that because if you programmed say in FORTRAN on a punch card and you had six minutes of compute time a day and you spend 20 hours writing your software and punching it into cards and then you submitted it to the mainframe and during your six minutes the mainframe would do bzzz error. Now you have to go back spend the day figuring out why, fixing it, punching it back into cards and putting it back on the mainframe during your next six-minute window. That’s how programming started.

And if you were a programmer in those days the attitude was “Thank God I don’t have to flip a thousand switches to do this. This is so much faster it only takes 48 hours to do one life cycle” and gradually this gap started shrinking and shrinking and shrinking. If you are really, really crap programmer you don’t even plan, design or think much about your software you just put some shit together, hit run, see what happens. Fix it, run, see what happens. Fix it, run, see what happens, reiterate really, really fast.

The really great programmers are not that sloppy they put some thoughts into every line they changed but all of us have moments when we start hacking, that’s where the phrase hacking comes from – from Berkley in the 1970s where that’s how they fixed software. He just hacked at it and then it works. But that encourages some mentality where your worst mistake lasts two minutes and no one gets to see it because you don’t ship that. That’s the beginning of a drift apart of two cultures.

In hardware your mistakes are immortalized on silicon and they might start fires. In software your mistakes disappear somewhere in a (0:07:19) nobody really has to look back at the previous (0:07:23) and see all of the horrible codes you wrote, right? We all do that. If you’ve done software programming that’s how most of it is programmed. 

That has some very interesting implications to the current debate because there’s another fundamental and really important difference in the culture between hardware and software. Hardware is all about ship date. You worked two years for that day when it ships and once it ships it’s out of your hands. You made the right design trade-offs are great. You made the wrong design trade-offs you’re going to be five years behind your competitors until you figure it out again but that ship date is the last date. And once it’s out there it’s out of your hands.

Look, recently at the perennial battle between AMD and Intel – how many of you are familiar with chip architecture in AMD and Intel. About two years ago, not too many so I’ll make it simple, two years ago AMD and Intel started working on their latest generation chips and having a five-year horizon they had to make some bets, some trade-offs, some design decisions. AMD made the bet that most systems are shipped have a graphical crossing unit and for advanced mathematical operations, matrix manipulation, floating-point arithmetic etc. you would have a development environment that would take that optimize into an open (0:09:08) and ship it to the graphics cards for processing because it’s a thousand times faster with that kind of work. So, why would you do that on a general purpose CPU? It didn’t seem sensible. So they decided let’s put 64 cores on a chip but only give them four or eight floating-point arithmetic units.

Intel said the software is not going to get that good, it’s not going to specialize these instructions. Let’s put just eight cores on a chip but give everyone of them a floating-point arithmetic unit. Two simple trade-offs, right? Two decisions. Is the industry going this way or is it going that way. Are we going to be able to leverage this new technology or not? Will the software catch up with what we’re trying to do? With the architectures of the future look more like this or like this? Agonizing over that choice they get a ship date in mind and they ship. Intel got it right, AMD got it wrong. Intel dominated the desktop and server environment for the cycle, right? Now, AMD gets to try to again four years later, three or four years later. That’s simple trade-off change the fortunes of that company and the direction of an industry.

We see this in other examples let’s go for bigger hardware. I am a fan of aviation. I’m a private pilot. I love that stuff. Boeing, Airbus five years or so ago they made a very important bet. Airbus said it’s mostly going to be about hub and spoke connections they’re going to have big hub airports and they’re going to run hundreds or thousands of passengers on single routes so what we need to build is a double-decker aircraft that can seat more people than ever before. And Boeing said no, it’s going to be mostly regional point-to-point and if we take an aircraft and allow it to extend its range to 12,000 nautical miles then that choice is a better choice and they built the Dreamliner. Boeing was right, Airbus was wrong. Boeing shipped a hundred times more Dreamliners than Airbus will never catch up, right? So, now they get to try that out again. There’s going to be a five-year life cycle.

If there is a bug in your software even in this environment you can fix it in a matter of hours, ship a new release, done. So, one important differences of this idea of ship date, once you put it out there you’re lock in for a couple of years and that makes you have a much more conservative attitude. But it has a good side because you will only encounter one of those ship dates a couple of times a decade, three time, four times a decade that’s it. With software however something else happens. You ship date is not the end of your problems it is the beginning of your problems because once you ship maintenance starts and very quickly it became apparent in the software industry that software behaves a bit like perishable produce. It’s got three days of shelf life after which you if you haven’t done maintenance it starts rotting on you, right? Software gradually degrades. Especially today’s software which is open source software in a very dynamic environment with lots of dependencies to third-party libraries everything is moving. As a cell (0:13:14) changes something we find a new bug, new release. The tolerance of the network change, new release. Berkeley DB doesn’t behave the way you expect it, new release. And so software is like a never-ending relationship which are trapped in like there is no ship dates, it’s continuous shipping, it’s continuous maintenance. There is no ‘and now we’re done.’ There’s only ‘and now our troubles begin.’ This divergence has created a massive culture difference between the culture of miners and the culture of software developers. It is at the root of the current discussion we’re having.

From the perspective of someone building hardware of course you’re not going to ship and you chip architecture to fix the problem just juice up the clock speed. This architecture still has plenty to go, right? It’s got room to grow, change the parameter for God’s sake, juice up the clock speed and we can keep the current architecture and just ship it.

From a software developer’s perspective if you’re looking at the bigger space it’s much better to change the architecture now before you have a loss of technical debt and software CRUD and accumulated UTXO and enormous Blockchain sizes on your data store that you have to keep forever and of course you’re going to be maintaining this either way might as well do an architecture change.

This is the fundamental culture difference between the mining community and the software development community in Bitcoin and not just Bitcoin. Bitcoin is just the one that has the strongest, biggest, most vocal mining community. But that’s not the end, that’s just the beginning because without even noticing we now have a completely new category which is going to create a completely new culture and that is trustware.

What is trustware? Trustware is this weird emergent phenomenon that happens when you combine consensus rules that are running and instantiated in software with the backing of hardware deployed on a global network with a diverse set of participants all of the headaches of hardware, all of the headaches of software and some new ones.

Now when you ship is really important because if you’re making a consensus rule change you have to coordinate an entire global network but the ship date is no longer the end of your problems, it’s just the beginning of the problems because now you have to maintain it forever and every mistake you make gets baked into the Blockchain and has to be carried with you in the consensus rules forever.

In Bitcoin there are no bugs, there are only consensus rules created through tradition.

So how many people here are software developers who’ve worked in Bitcoin? All right. You probably know about this one it’s a classic.

So when you write a multisignature script and the code gets to the part where it says up check multi-sig verify the code has to go and pop as many keys off the stack as you’ve defined as the last parameter N, right? And then inspect the signatures which should be M (0:17:27) and do something. It turns out up check multi-sig verify pops one extra value. That’s a bit of a problem because if you do a multi-sig of three keys and it pops four things there aren’t four things on the stack and if there aren’t four things on the stack you get a stack error and your script crashes and your money is no longer spendable.

Now when then bug happened because it did happen accidentally back in probably 2011 it wasn’t fixed before some people put spendable Bitcoin and redeemed it on the script on the Blockchain. And to do that what they did was they put what’s called a null value, a dummy value so they go okay, you want to pop four things off the stack three of which have to be keys and one which is going to get ignored here’s (0:18:29) key, key, key and so you pop (0:18:34) key, key, key. Throw away the (0:18:35) keep the three keys it works. Done. But now that script has to be valid forever because every note in Bitcoin validates everything forever. Oops!

Now, developers are writing new versions of the software multi-sig is now part of Pay To Script Hash formula. Guess what, it still pops an extra value and so I write in my book there’s a big old notice it says you will see an extra value in all the redeemed scripts that’s because there’s a bug. That bug cannot be fixed. It’s with us forever. Why can it not be fixed? Because the fix is worse than the problem. I mean sure you could fix it. You could just put a thing in the code that says from now on just pop three and now everybody knows (0:19:34), right? Redeemed scripts that just popped three, fine, no problem. But now you have a piece of code in your Blockchain what is effectively a software that says before block X pop four, after block X pop three if you see this script so that all of the scripts that came before are valid and all of the scripts that will come next are also valid but don’t need that dummy value. What you’ve done is you’ve moved the CRUD from your script into your code and now these two, three lines of code need to be maintained forever. What if there is a bug?

If you write three lines of code on average one of them is going to be wrong. So for every line of code you add to the consensus rules you got to make sure you don’t had a bug. How do you coordinate an international network so that everybody make sure they change the rules at the same time and you don’t accidentally invalidate transactions? This is the essence of trustware.

The essence of trustware is we are now writing software that gets backed by hardware deployed on a network and establishes a set of global consensus rules which if you make a mistake on this consensus rules and go out of consensus you can lose millions. You can get cheated out of transactions. You can be suffering replay attacks or (0:21:09) attacks or all kinds of other attacks. This is not a game. This is a new software frontier only it’s not software. It’s trustware. And trustware is way more complicated than software or hardware or software and hardware put together because it’s also a global network component that’s controlled by independent actors.

Why on earth would we do all these? I mean it doesn’t sound like a fun development exercise. Why are we creating this thing that will have its own culture, trustware developers, consensus experts that requires its own deep understanding and analysis and review? Why are we doing this? Because it gives us something amazing. It gives us a decentralized platform of trust that is neutral and not controlled by anyone and that’s worth it. But it’s bloody painful.

And so, this is where we are today within this network especially in the case of Bitcoin we are now seeing a direct conflict. It’s not a violent conflict, it’s simply a conflict of ideas. It’s a disagreement about the future of the network. It’s a disagreement about the future of the currency and the future of the consensus rules. I have to assume good faith. I think both parties see the way forward as the best way forward for Bitcoin. But in technology it’s not just a matter of opinion, there is truth. Truth means something. There are correct opinions and incorrect opinions. There are opinions that matched the facts and once it don’t it’s not a system of belief, it’s a system of science.

Unfortunately most of the conversations that’s happening really looks like a system of belief or more like a competition of soccer. So, on the one hand you have die-hard fans of FC Barcelona and on the other hand die-hard fans of Manchester United. There is no right or wrong. There is no correct answer. There is only my team and your team and your team is wearing the wrong color, they look silly and they can’t play soccer. Clearly any intelligent person can see that. Unfortunately that doesn’t leads to any scientific conclusions which is why we’re here.

So, what we’re seeing today is the combination of a fundamental culture clash between people who primarily build, manage, deploy and run hardware and people who fundamentally build, manage. And maintain software and that’s why you’ll notice some Chinese people on the software side and then some Westerners on the hardware side and this is not a culture clash between East and West. It is a culture clash between hardware and software and from within that a new culture is now emerging. A culture of developers who are building trustware who are gradually seeing the nuances and incredible difficulty of building a system of consensus rules that is backed by hardware and deployed on a global network and that is trustware. Thank you.

Written by Andreas M. Antonopoulos on March 25, 2017.