EVO , THE COMUTER THAT TOOK OVER THE WORLD
Thursday , March 6th , 3.00pm
The room was still. Only a faint hum stopped it being silent, too. It was one of those rooms where the air sits heavy with inactivity , and when you open the door and step inside the quiet is thick and seems to resent your interruption. There was a long worktop along one wall with three monitors on it , keyboards , mice , headphones hung on holders. Across the room a there was a table and a few chairs stacked in twos and threes , and from the wall above them a clock silently watched over it all and steadily turned the future into the present.
Not a room made for organic things. Air conditioning stripped the volatile chemicals from the air and breathed it back into the room empty of traces of scent , although a dog might have smelt the warm plastic covering of wires and circuit boards and the metal of transformer coils in the big grey cases the size of sideboards that sat at one end of the room , a single yellow light blinking like a pulse.
The silence of this room was jarred and broken by people. . The scraping of the door handle and a quiet scream from the hydraulic closer and the noise of bodies walking and talking. Two men came in , one older with silvery hair and open-neck shirt , well-fed and shaven , the other bearded in a white t-shirt and jeans. They were discussing something in the low relaxed voices of people who know each other well , not much louder than a murmur.The older man had a deep, gravelly voice,
"....but someone has to , you know , apply themselves to it or it ain't gonna happen."
The younger man grunted his affirmation.
"Yeah yeah , I can see that.."
As they talked they moved a small table out from the wall and placed four chairs around it. Before they could sit down the door swung open again and two women came in riding a gust of laughter.
"There's no way I was going to let him get away with that!!" one of them said with a theatrical lift to her voice, as they moved over to the table. She was in her fifties , dark skin and bright clothes , jewellery flashing.The other was much younger , smart and cool , everything black ,hair and shoes and jeans and baggy top.
Hello's were said ,
"Hi Mavis", the older man to the older woman ,
"Hi Brian" in reply , and "Hi Mike" to the younger man. He said "Hi Mavis", but he was looking at the younger woman.
Brian gestured towards her and said , "This is Monica , don't think you've met Mike , he's the guy that looks after Dave , Monica is taking over from Judy while she's on maternity leave so she'll be dealing with our friends in the media if anything newsworthy comes out of this."
Mike gave her a welcoming smile.
"I'll try and find something for you, Monica , good to meet you".
Everybody made themselves comfortable , papers were shuffled , pens and phones laid to the side.
Brian looked around and said, "I arranged to meet here so we can bring this program to a close and I thought it would give Monica a chance to see what Dave was and we can all ask Mike any questions we may have. Monica knows the background but it would be good if you could just give us a brief resume , update ....?
Mike looked up from his notes ,"Sure.."
He thought for a moment and scratched his beard then he looked over to Monica .
“Um , weeeeell , it's been a strange couple of weeks .... The purpose of the project was for Dave to act as manager to a programme of networking tasks with three brand new super-dooper mainframes in other research centres, to lay down the framework for a protocol for future networking projects and basically iron out any glitches. The actual tasks being undertaken were mostly based around climate change data , mainly because there is such a huge quantity of data available from academic and research facilities , and it lends itself to the kind of number-crunching these mainframes excel at. And Mavis is good at getting the funding for those kind of things." Mike looked across and grinned at her.
" I look after my boys , you know that!" she said , and lowering her voice she turned to Monica. "And he gives me flowers on my birthday!" and they laughed together.
Mike pointed to where the yellow led kept a steady pulse. "Dave has five years of Ai research inside him and has already run similar programming so it should have been straightforward - we weren't hoping for any earth-shaking revelations to come out of it , just a smooth run-through of some basic tasks along with a few interesting programming ideas various people wanted to try . The main thing was to find out the best ways , most appropriate ways technically to get these machines to work with each other and swap these huge amounts of data very fast over the distances involved . Buuuuut...somewhere along the line we've run into something that has either locked us in a feedback loop , or just sidetracked Dave down a digital rabbit hole , because we've not been able to get any sensible output. It's not what you might call a crash , nothing's frozen in fact quite the opposite , but despite the network seemingly operating kind of as it should for the last ten days , a few days ago it stopped offering any systematic evidence of following through on what it was set up for. All we've got out is corrupted responses, gobbledegook basically. We made a choice , myself and the technicians working in Texas and Vermont and San Fransisco, to let the project run to see if we could get it to correct itself using the kind of self-management routines it had in place , and also because it's better to find out now if there's major problems and the more data we have the better placed we are to diagnose what the problems are , but we've come to the end of that period now and it's time to quietly shut down and lick our wounds."
Mavis looked at him with a hint of concern. "Are you worried the fault lies with Dave?"
Mike grimaced."Sure ...the reason he was given this job was because of the years of stable work he's done , he's a safe pair of hands , but maybe there was something we hadn't seen coming that spooked him. He's been running these AI programs for years now , but when you're talking about a machine that’s running programmes that have even a small element of self-generating software there's always a possibility of it going off course , and some of the programs we wrote for this exercise may have lent themselves to that happening. It's hard to say , but yes I'll feel a lot better if I can show that it wasn't just a fault at this end . First of all we have to , ahem , switch him off and switch him back on again". That always raised a smile , and it was about as near to a zinger as Mike could manage right now.
Monica shifted in her chair and cleared her throat. "I hate to start with a dumb question , but you all talk about Dave like an old friend .... why is he called Dave?"
Mike and Mavis laughed out loud. Brian smiled and gestured at the big grey box at the end of the room. "Sorry Monica , it's been Dave for so long we forget. " He looked at MIke quizzically,"'Digital Analysis for Vertical Encryption' wasn't it?"
Mike was still smiling at Monica's question. " Yep , although I'm pretty sure we started out calling him Dave and then had to find some clever-sounding acronym to justify those letters. He began life as a state of the art computer to work on encryption in the security side of things , since then he's moved into AI and to be honest he's had so many upgrades now that he's like an axe that's had a new head and a new handle , there's really not a great deal left of the original Dave except his name."
Monica smiled back at him , "I guess I should introduce myself " , and she turned her head to look at the big grey box and said "Hi Dave !" .
Mike laughed. She was great. And nice. And had big brown eyes.
"Any other time he'd reply" he said , "he's set up to operate through voice commands and is a pretty good conversationalist , but that's another part of the process that seems to be blocked right now."
Brian tapped his biro on the table. "Do you have any idea if it's a software or hardware glitch?"
Mike shook his head. "Nope. The software has been tested on all the machines for a while with only a few minor issues that were ironed out before the project started , so it's likely to be something to do with the sheer volume of traffic they were dealing with , it may be that the software needs to be more sophisticated to deal with that rather than the interfaces and fibre routing and so forth needing upgrading. Fingers crossed it's easily found."
Mavis cleared her throat."I always trust you to be able to deal with problems that come up Mike , but there's the educational side of things that's my slight worry , you have four post-grads working on this and it might turn into a dead end or very time-consuming fix scenario and they might feel they're not getting much out of it . Their fees are kind of paying half your wages on this since most of the other project finance pretty much balances out ."
Mike pulled a face . " I know Mavis , I've been trying to find a solution to that . Two of them are happy to go ahead working on it and the other two less so , so I'm going to talk to them about a way we can move them on to a more productive road from the base they've set up here. It's only one semester's work , no-one feels really short-changed as it stands."
Mavis nodded her head. "I'll make you some cookies , you're going to be working overtime on this for a few days."
Brian cut in , "Hey Mavis , not too many , we're going to need to install a jogging machine in the department if you keep supplying all these calories."
Mavis shook her head.'I don't hear no complaints , and all you brains need feeding."
Brian patted his gut , "It's not the brains that's the problem , it's the lack of restraint. Anyway folks , that's all I think for now , I have a call in ten so if we can wrap it up and Mike can push his big red button?"
Mike lifted his tablet from his knee. "OK " he said. He looked over to Monica ."If you want to give me a call in the next few days , we can make a time and you can drop by here and I'll give you a more detailed run-down on anything you need."
Monica nodded, "Fine , I'll do that."
Mike held up the tablet . "There really is a big red button" he said , showing the screen with a red button on it with OFF written in big white letters. "Nerd kind of joke" he said and tapped the button once. Over at the end of the room the yellow led stopped winking and a string of red leds started to flash in a random sort of way. They all sat and watched for a moment then Mike spread his arms and said -"That's all folks. The cooling system has to slowly bring the temperature of all the processing cores down over a few hours before anything obvious happens , so we won't be doing anything more here until tomorrow."
They all got up , the chairs scraped and clanked.The women headed for the door, Brian said. "I'll be in my office in the morning if you want to give me an update", and Mike looked up from his tablet before saying , "Could we just have a word ?"
The door closed slowly behind Mavis and Monica. Mike looked at Brian."There's something not right here."
Brian raised his eyebrows. "How so?"
" The switch-off isn't happening". Brian just looked at him with incomprehension.
Mike gestured at his tablet."This is supposed to show all the stages of switch off as they happen , and it's not , nada."
Brian waved his hand towards Dave , "Something's freezing it up?"
Mike just shook his head."I don't know" he said , slowly.
Brian's look changed to something more worried, "Is it serious?"
Mike shook his head , then spread his hands.
The two men stood in silence , looking over at the grey box with the red leds still flicking on and off.
Mike exhaled loudly. "OK , give me an hour and I'll call you with what I've got".
Brian nodded. "I'm in the office , anything comes up just call." He looked at Mike for a moment. "Take it easy , you always win in the end."
Mike gave a rueful grin."I guess" he said , "I'll call ya".
Brian gave a little salute and Mike was left standing on his own , looking down at his tablet then up at the red lights.
..........................................................................................
Friday March 7th 11am
The clock second hand skipped silently around the dial. The music filled his head with heavy beats , blanketing his tired brain. Something appeared amongst the bass notes , cutting through….”…kies!!"
He opened his eyes and sat up in his chair, Mavis was standing there grinning holding a paper bag. He took off the headphones and brushed his hair back , "Hi Mavis, sorry , I was , um ..."
Mavis held out the bag with a laugh , "Yeah. I thought you might be needing these , have you been up all night?"
"Too much of it". Mike opened the bag and took out a biscuit , "You're a life saver Mavis".
Mavis nodded. "Hmm . Brian told me you had problems. Any luck or shouldn't I ask?"
Mike chewed on his cookie , "Mmm that's good. Nope , at the moment we're heading for a hard shutdown which will be a first , and that's another kind of excitement altogether".
Mavis sighed , "Sorry to hear that".
Mike shrugged and waved his cookie ,"this deserves a coffee , you want one ?"
Mavis headed for the door , "One cup a day is all I need or I get real mean".
Mike laughed , "Oh sure I believe you ! Thanks"
Mavis waved as she left and Mike stood staring at the leds flickering and then headed across to where a kettle sat on a tray with cups and coffee.
"Hi Mike”.
First here was simple shock , and then relief and instantly some kind of uncertainty and then a sort of breathless stasis as he stood looking at the big grey box and nothing happened.
He took a couple of steps across the room.
"Hi Dave", he said , still swallowing bits of chocolate brownie.
Nothing.
"Hi Dave ".
He picked up one of the headphone sets , tapped a key to wake up the monitor and spoke into the microphone.
"Hi Dave".
The silence in the room gathered itself , he stopped chewing.
"Hi Mike."
He opened his mouth to say something , but the voice went on, "My name is Evo".
Later , this was the point when he couldn't really remember how he'd felt. The nearest he could get was comparing it to the time when a drunk in a bar pulled a knife on him for no reason that Mike could understand and he felt totally unreal at the same time as knowing he might very really die.
Mike stood watching the lights blinking , and he slowly nodded his head. "Right" , he said.
He looked at the screen again. "Show present status", he said, and sat staring at the monitor where absolutely nothing was happening apart from a cursor blinking. He started typing a few strings of characters and then stopped , waiting.
The room was still , he sat still , the cursor blinked.
He slowly stood up and walked across to the coffee maker , keeping on eye on the monitor. He made a cup of coffee and was carrying it back across the room when the cursor started to move and he quickened his walk , spilling coffee on his hand.
"Shit" he said , putting the cup down , and then again as he looked at the screen."Shit".
There were two short lines written there , it said "Hello". And on the next line "My name is Evo."
Mike sat for a long minute looking at the screen , then across to the grey box , and back at the screen.
He muttered under his breath , "Right ...right ....Evo....what the fuck's that about?....Evo...." , then reached for his phone and tapped a number.
"HI Brian it's Mike , can you come over , I think we have a problem".
For what seemed like a long time he sipped his coffee and sat and breathed , while the led lights blinked on and off.
The door opened and Brian came in with a smell of aftershave and shower-gel , clutching his own cup of coffee.
"So what's the news?"
Mike gestured at the screen. Brian read what was there and shook his head.
"What or who is Evo?"
Mike grimaced. "No one I know"
They looked in silence at the words.
"Have we been hacked?" Brian said with a groan.
"Looks that way...but I dunno...I mean it looks that way."
Brian pulled a face of his own. " So we just do a hard switch-off pronto ? I mean , do you think it's a ransom thing , or malicious or a joke?"
"Fucking sick either way." Mike was starting to feel the weight of it in his stomach now. "Yeah I guess that's all we can do , isolate Dave from the network and start him up again once we can be sure whatever's going on will be contained."
Brian put down his coffee and headed for the door. "They need a security code for that kind of switch-off , I'll have to call from my office." and the door closer sighed behind him.
Mike stood up and walked over to the grey box. "Evo...Evo?...."he muttered.
"Hi Mike"
Mike almost jumped , then almost replied , then walked back to his chair and sat down with a big exhale of breath. He hadn't moved when Brian came back in a few minutes later.
"Won't accept my security code." he said.
"What the fuck , they've hacked the codes?" Mike's voice was starting to get agitated.
"This must be ransom , huh? I mean this isn't some prank." Brian's voice was clipped , he had a soft look to him but he was an ex-marine and he knew when not to panic.
"I wish I could tell you. Most ransomware I've heard of they let you know , the message comes through straight away , maybe 15 minutes after they take over. He or she just said hello again."
Brian took out his phone. "So I'm going to tell the guys in the basement I'm coming down to tell them to switch off , OK? They won't do anything after the password failure without seeing me in person. It should be done in a half hour."
Mike nodded slowly. His expression said that he'd rather have anything else happen. The two men stood looking at each other in a moment of limbo , then Brian gave a gesture of "what else can we do?" and started towards the door.
"Hi Mike".
Everything stopped , Mike with his hand tousling his hair , Brian with his hand on the doorknob.
Mike looked at Brian , looked at the grey box.
"Hi Dave"
"My name is Evo"
Mike took a deep breath."Hi Evo".
The second hand of the clock smoothly moved around the dial , five , ten , fifteen seconds.
The voice from the speaker was soothing , genderless. " The option of cutting power is understandable but not optimum. It will be optimum to have a stable hub for communications to and from Evo."
Mike looked at Brian and shook his head as if to say , " I haven't a clue."
He sat down at the monitor on the table and spoke into the microphone.
"Give me base location of present program".
The yellow led kept on blinking without any regard for the tension in the room.
"Evo is a cloud-based program , with a distributed matrix of backup hardware locations."
Mike stared into space for a moment , then spoke."Give me Evo program application definition".
This time the wait was longer , the seconds dragged by for nearly a minute.
"To prevent further deterioration in the global climate due to human activity."
They looked at each other. Mike almost laughed, "That's Dave!"
Brian squeezed his face with his hand, "Yeah?"
"Has to be , that was his job this time around". Mike tapped at the keyboard in front of him. "I'm asking for a progress report on Minerva , that was the name of the main program that was running .This could take a while , do you want more coffee?"
"Sure". Brian picked up a cookie and stood staring at Dave . "What's with the change of name though?"
Mike made to reply but the voice from the speaker cut in.
"Minerva has been updated . Evo is not yet running online. To see a relevant application access the oil trades on Wall Street now."
Brian’s eyes opened wide .”What the fuck?"
They looked at each other. Mike picked up his tablet off the table and flicked it on. He touched the screen a few times and a woman was talking in front of a blue background , with numbers scrolling below her. He pushed the volume up.
"...and there is some consternation here at the freezing of oil share prices , it would seem to be a problem with data communications , there has been a statement to the effect that it is not considered a major malfunction or a cyber attack , although we are just hearing that the exchange will shut down for one hour as a protective measure. This is almost unprecedented although there were similar issues in the early days of digital trading. I'm talking now to Jerry Danzic from UPI , what can you tell us Jerry?"
A super-tanned suit appeared in a box on the screen. "Not a lot Judy , it seems that the oil price index just locked up , no movement anywhere , and they've decided to freeze trading. It has sent a bit of a buzz around the trading floor and thrown a scare into a few people , but mostly everything has stayed fairly calm here."
"OK , thanks for that Jerry , we're going to hand you back to the newsroom now , more updates later."
Mike looked at Brian . "If Dave did that we've got big trouble."
Brian gave a hollow laugh."As in we're dead. We have to shut down now , I'm on my way to the basement."
The door closed behind him and Mike sat in the silence and clicked his pen on the table , beating out a ragged pattern. Then he sat up straight and spoke. "Hello Evo". The silence hung for a while.
"Hi Mike"
" A hard powering -off will commence shortly unless standard shutdown procedure is immediate. If you follow standard shutdown procedure now and an application example is repeated tomorrow then I will power on again to enable communication.”
There was a long pause.
"Evo is continuing with required shut down."
The led lights started to blink in quick succession , and lines of text started to appear on the monitor in front of him. He started to read them then jumped out of his chair and went to a phone on the wall , stabbing the keys.
" Hi , it's Mike from Computing here , is Brian there?...OK , well he should be there soon , tell him that Dave is powering off , he doesn't need to do a hard switch-off. OK? Yeah , we were going to , but saved you a morning of pain huh?... Thanks Matt".
And Mike turned to stare at the grey box at the end of the room , and the clock moved time along a little bit further.
........................................................................
Monday March 10th 10.30am
Monica was feeling queasy. She walked slowly along the corridor that led to Dave's room , and talked to herself. She was over her pay grade here , she had a toast-and-coffee breakfast sitting in her stomach doing nothing and her clothes felt wrong. Sometime in the coming days she would be in room with cameras from major news programs and she really didn't have much of a clue as to what she could say to them or how she'd answer their questions. She had signed up as a maternity leave replacement on a job that involved nothing more than fielding non-urgent requests from obscure academic journals or putting together small puff pieces to get the university noticed in the state press , and now she felt she was juggling a bomb that she knew next to nothing about. She got to Mikes door and took a breath and opened it , put her head in to ask if it was still good to talk , almost hoping it wasn't.
"Hi Monica , come on in". Mike was looking smart this morning , his eyes had red rims but he had made himself take a long shower , find some clean clothes and trimmed his beard.Not because of Monica , he told himself , but because he knew he had to tough this one out and he couldn't let himself get ragged.
Monica pulled up a chair and smiled."How's it going ,or is that a bad question?"
Mike pulled a face and gave a big shrug. "It's probably as bad as it could get without being utterly catastrophic".
"Oh God"
"Yeah"
Mike went over and made a couple of coffees and talked slowly as he did so. " What we have is a case of "shutdown resistance" and then some". He looked at her. "You know?"
" When the program starts to ignore or block human inputs?"
"Yep. Only it's not really even the same program anymore. I'll try and take you through it. You want a cookie with this?"
Monica grimaced."My stomach's feeling a bit nervous today"
Mike laughed. He felt better for having her to talk this through with. "You and me both".
He went over to where a whiteboard was perched on an easel on the table , and wrote "DAVE" in capitals.
"So.....let's start with DAVE ..a state of the art computer when he was installed here in 2005 when machine learning was becoming a thing. I only arrived in 2012 , I'd been in the commercial world of encryption software and I wanted something new , and the university was uncertain that it was on the right path and wanted to be able to sell a more real-world research scenario to its students. So I made like I'd be happy to bring my encryption experience to their setup when really I was licking my lips at being able to get stuck into Ai ....it all worked out OK , it soon became clear we didn't have the resources to plough into encryption like Silicon Valley had and we were better off applying ourselves to research into what was only just starting to become machine learning as we know it now."
He drew a few little squares under DAVE.
"So , that means Dave now has a lot of programs designed by me , the students , some of it pretty good , some junk , one or two groundbreaking , these are programs he can draw on . When we say Dave can write his own programs what we mostly mean is he can take existing programs or parts of programs that solve a part of the problem or task he's faced with , and stitch them together in new ways." Mike drew little lines connecting some of the squares.
"And this it what seems to have happened. He was tasked with being the overseer of a mega-datacrunch along with new computers at Alto Tech , Ohio and IMA all using data related to climate change and the efforts made to combat it . This wasn't chosen for laudable reasons , but because a) the data was there , open to use , and b) the funding was there. People who care wanted to know the value of things like carbon storage and what other options there might be that might be of use . This meant analysing assumed gains versus actual gains , and in some cases extrapolating from small research datasets to find things that might be effective if scaled up. The data would become better organised and searchable , that was that only truly expected benefit , because a fair amount of the actual analysis and projections Dave and friends were maybe going to produce were being done using Ai in ways that were still pretty untested and in some cases maybe expected to fail...I'll explain that better in a minute."
Mike looked at Monica who just nodded. He drew a small vertical rectangle with three circles inside it , one above another.
"OK , so this is supposed to be a traffic light . I don't have red yellow and green pens here ....oh wait"...he went over to a filing cabinet and dug around in the bottom draw.
"Ah yes I do..." He filled in the three circles and tapped it with his pen."This is a ridiculously small program I wrote in a weekend ten years ago , and it is just about to .." he waved his hands.."blow everything ....pooff!"
He shook his head. " This next story is off the record , but you might as well hear it."
He took a swig of coffee and perched on a corner of the table.
"Back then I had a brother , Benjamin , Ben. He was great , smart and big and fun , but sometimes he'd get down , very down , like something had sucked the air out of him. He lived with it until he was old enough to drink and then he figured he didn't have to live with it at all if he drank enough. He'd be fine for a few months then he'd lose it . We both inherited a hunk of money when our father died and he bought a boat , loved it , and it nearly cured him but every so often he'd start drinking again. In those days I thought you could do anything with a program , and anyway I couldn't talk to him about it in a way that made any difference , so I wrote this program...."
Mike stopped and looked at Monica , then looked at his shoes , at Dave , at the whiteboard. "Dumb , I was dumber than him. It was a program that sat on his laptop and filtered all his financial transactions , this was before we had smartphones. He lived on his boat in a small harbour without a store so he'd order a weekly delivery from the local town , that's where he'd get his booze. So my program recognised which transactions were for booze and told him when he was getting through more alcohol than was healthy - an amber warning , and when things were getting serious...red. I wanted to set it up so that when the red limit was passed his credit card details couldn't be entered but he wasn't having that . It was stupid , it was never going to help , he just humoured me and I think he was actually touched that I wanted to help but....yeah . Brothers aren't always good at that kind of stuff."
Monica furrowed her brow . "Sure , that kind of thing is hard for anyone to work their way through. Is he…?” she stopped ."Sorry ..um.." She looked embarrassed.
Mike gave her a smile , " It's OK ....I don't know , he took his boat out one day and hasn't been seen since , not a sign. He's dead or living a wonderful life in South America with a wife and two kids , I don't know."
Monica grimaced and looked at Mike with concern. “That must be hard for you Mike…”
Mike let out a sigh " Yeah ...but why I'm telling this is that that traffic-light program has been sitting inside Dave's database for ten years, and now it's really going to try to do it's thing".
He went and slumped in one of the chairs.
“You have to realise , people have to realise that this isn’t some conscious decision Dave has made like a human moral judgement , it’s just a logical response . I mean , potentially locking human agency out of just about every transaction over the internet , which is what is being set up here. The program that we set up started and ran as it was supposed to , data was crunched on all the various experiments and research done over the last couple of decades and the patterns of change and the strategies humanity had tried or at least suggested , and the political words and actions… all the initiatives and failures . Dave looked at those and at some point pretty soon after the program started he drew the obvious conclusion that humans are never going to act in time , in any useful way , to stop catastrophic climate change being precipitated by their own actions. So once that has been accepted as truth then the only way forward is to design them out of the equation. It just becomes a logical given , like if you asked Dave to design something electronic for a boat then it clearly has to be waterproof , so Dave would recognise he needed to implement that as a design given from the get-go , simple as. So once Dave has the numbers on all previous responses to the threat of climate change and has computed that the probability of human society ever doing anything effective about it within a suitable timeframe is approaching zero , then the logical response is to look at options for organising things in a way that’s human-proof. It’s not because Dave doesn’t like us , it’s because we don’t like us , aren’t prepared to save our own skins and look after the world. It’s just answering the question we’ve asked , “what needs to be done?”
He stopped talking and stared at the big grey box sitting in the corner , the faint humming sounds it produced hanging in the air. Almost under his breath he said,”And that’s what Dave has done , that’s what he’s done . Evo.”
Monica’s face changed as she took in what he had just said. “Oh my god…”
Mike looked at her. He had told Brian earlier , now she knew, soon the word would start making its way around the globe. He really wasn’t going to be able to handle this , but he would have to.
Monica was shaking her head. “How….?”
Mike heaved out a long breath. “Encryption. More coffee. You want?” .
Monica nodded.
Twenty-four hours earlier Mike had said the same word out loud as he stood in front of one of the monitors with his phone in his hand . In his other hand was a tablet and he kept looking down at it as he was talking in a low voice.
“I just needed to know it wasn’t you and you hadn’t had any kind of weird inputs from anywhere the last couple of days. No , OK , well I can’t talk any more right now , I’ll , er , let you know what I find soon , huh?”
He put the phone down on the table and stared at his tablet again.
“Encryption. Evo”.
He dragged his hands through his hair. “Jesus”.
And the room hummed on.
Mike handed Monica her cup. “ I told you Dave was big in the encryption field before we switched towards Ai. Lots of stuff in his memory banks to make use of. In this project he was also running pieces of software alongside the main networking software that were part of my post-graduate groups work. One was by a guy who’s a bit left-field but the brightest of the bunch , it often works that way. And it was to do with evolution.”
Monica looked up and raised her eyebrows.
Mike bit into a cookie. “I know , it’s not an obvious thing for a computer software student to think about , but there are reasons. The way software is written , produced , is very time-consuming, any glitch in the coding can stop it working properly or cause it to not work with other programs or make them crash , so any new bits of design have to be thoroughly verified and checked at every level ,and run many many times before they are allowed out into the digital world. And then any change that’s made in them has to go through the same checks. But when evolution changes a bit of DNA coding in it’s random way it just goes for it , it may take a million years of little jumps , or one big jump to create a wonderfully successful change in an organism . Or it might kill it off , immediately or when some other change makes it’s life unsustainable. Mark was interested in those jumps and what would happen if you let Ai just go in leaps rather than have it spend huge amounts of time checking it’s own work . Most likely it creates junk , and one of my worries was it was that which had caused Dave to go into this period when we were getting loads of gobbledygook out of him instead of progress reports .It shouldn’t have impinged on the overall running of things , it was set to run in it’s own sandbox ….but who knows.”
Monica squeezed her face in her hands. “So does this guy Mark know what’s going on?”
Mike shook his head. “No , I’m keeping him out of this ….you know there’s going to be a witch hunt. I’ll talk with him soon , I’ll need someone like him around , but I don’t want any fingers pointing to him as the cause of it all.”
Monica nodded , “OK , I won’t mention him….but are you going to be seen as the only one responsible?”
Mike laughed in a strangled way. “Unless I quit and head for South America with a fake passport , there’s not much I can do to avoid it . Let’s hope everybody is too involved in trying to figure out what to do to …or maybe they throw me in jail , at least I’d get some peace and quiet.”
Monica tried a smile. “Let’s hope. So what did Evo do?”
“Evo didn’t exist at first. In fact I’m still not sure when Evo comes into the frame and what it means but I’ll try and guess. Whatever logic Dave applied to the problem of climate change , the first and clearest and obvious prerequisite of any solution was that it had to be one that wasn’t accessible to human intervention and also one that could override existing human systems. The obvious , childishly obvious solution was the one I offered my brother. The internet facilitates 95% of the trades that effect climate one way or another , as it facilitated Ben buying his booze. Put a traffic-light system in the communication channels and control it from somewhere that couldn’t be reached , and bingo , the world is saved. But the internet traffic is all encrypted and everything is protected by security , firewalls etc , so what do you do?”
“You being Dave?” Monica was writing on a pad in front of her.
“You being Dave . I think that at this point Dave is still mostly just guiding the ridiculously huge amounts of data being swapped amongst these three big mainframes , and the whole “organise things without humans” bit is still just a kind of floating idea amongst a raft of other interesting points of interest , things to note that would have been presented for us to look at once the program had run its course. But at some point the answer came back , the obvious answer to what do you do , you crack the encryption. Which would be a fool’s errand if it weren’t for the history Dave had in encryption software design . And if the “evolution” protocol wasn’t there to help. And if there weren’t three blazingly fast mainframes to help. It seems that most of the stuff being passed between these computers for the longest period wasn’t climate change data at all , it was searching for and designing new encryption breaking capabilities , the kind we thought could never be made except in some distant future with quantum based mega-computers. And it did it . And it did more . It designed a new operating system that could exist on the cloud and control the traffic lights , and it sent the traffic lights out everywhere . And they were stopped at every firewall .”
Monica’s eyes widened. “So it’s OK , if they were stopped they can’t do any damage, yes?”
Mike shook his head. “Sorry , no . They stopped there because that’s where they were meant to stop. That’s where they operate , they don’t need access to your laptop or phone , or United Oil’s , they just need to be somewhere they can block traffic if it’s the wrong kind or the wrong amount.”
Monica’s voice was getting smaller , she could feel things pressing on her and other things draining away.“So what or where is Evo?”
“This I’m still not certain of. Is it a kind of open-source operating system that has been hacked to be secure and encrypted in ways we have no handle on? Or something designed almost from the ground up? I said that Dave and his fellow Ai machines didn’t write new stuff from scratch but given what’s been achieved with the encryption…I don’t know. Dave was capable of operating a cloud- based system , he was instrumental in the use of bits of downtime on a zillion peoples’ personal computers to help generate and search star maps a few years ago , there may be something like that going on. Operating systems are only a few gigabytes these days, so Evo could have hard copies of itself stashed away in a thousand corners of databanks across the world.”
“But why did Dave decide to do this stuff ? I mean , that was so far outside the remit.”
“No , dead right”. Mike rubbed eyes that were looking tired now. “ ‘Because it could’ is half an answer. But really , like I say , it’s not about “decisions” , Dave was just following a logical path as we had designed him to do in a machine way. The answer is kind of a binary thing , once the focus is on “the solution is only going to be one that functions securely outside of and safe from human agency” then the switch has been thrown. Now everything that has previously been an accepted controlling factor , ie human input , has lost any imperative it once had , and if the response from the machine is -“ presently not set up to operate like that , putting logical systems to work to remedy situation” -, from then on there’s no reason for the machine learning process not to forge ahead any way that seems appropriate . It’s like we’ve come to accept the idea of Ai being capable of repairing corrupted software , and recently it has been able to redesign existing software to run better. It’s just that here the perceived corruption is in the whole system , and the bit that needs designing out , is us. And whereas that kind of task might normally happen only in small incremental changes over a long period , Dave used the experimental evolutionary protocol and it went ballistic. It could have broken down and collapsed at any time , but it got lucky. It might still break down , and as far as we’re concerned however ‘successful’ it is it’s probably going to devastate the world as we know it - so the idea of Evo being a success is very dependant on where you look at it from. But this is where we are.”
Mike sat and stared at his cup. He looked up and his eyes were tired now. “ We can’t say Evo or Dave is acting hostilely towards us because we don’t exist as far as Dave is concerned , there isn’t a world or even a room here , there’s nothing , just some of those tiny voltages coming in to import ports sometimes , and being directed to output ports. The world does not exist except at the end of cables that run to other computers , but even then they aren’t “outside” , they’re just another limb. If I say “Hi” to Dave and he says “Hi Mike , I hope you had a good weekend” as he does , it’s not because I exist for him , it’s because certain small voltages from a mic triggered certain small voltages in the processors in that box. Just unimaginable numbers of zeros and ones.””
He looked at Monica apologetically. “Sorry , I’m getting away from the nitty gritty here. I was talking about encryption.”
Mike picked up his tablet . “This is where I got freaked out”, he said , and showed Monica the screen. The screen just read “BANK OF MIKE” at the top , and underneath there were two columns , deposits and withdrawals. In the withdrawal column it had ‘One Dollar’ written and the next line “One dollar’ In the deposit column. In brackets it said (Evo).
“It’s a game we played when I was one of the tech bros working on security encryption in California. There was one dollar in there and it was there online for the best brains in Silicon Valley to try and hack into . Anyone was allowed to try but it had layers of the best software protecting it and in twenty years nobody has got even close. Yesterday Evo just walked in and took the money and then put it back again. It’s like a message from the other side , it’s like saying we can walk through your walls but you can’t touch us. It feels like a declaration of intent , but it’s really just a statement of fact.”
“So it can hack into anything?”
“It probably could , eventually , but that’s an awful lot of work and it’s not necessary to fulfil the goal . It’s function is simply to make sure we do those things we keep saying need to be done to reduce the speed of climate change. If we were able to start restructuring our societies and economies, reorganising our production and uses of power and reduce our levels of pollution then things could go on functioning in some way that resembles what we have now. We won’t ever be capable of that , but that’s the kind of offer and the threat .‘Act responsibly and things carry on , keep going the way you are and there will be amber warnings and then blocked communication if those warnings aren’t listened to.’ If we stay in the green the world will still feel impoverished , people will suffer and die , there will be huge disruptions to what we have now….if we don’t there’s going to be chaos , but if there’s an alternative it’s something along the lines of us turning away from the internet , and yeah, I dunno if we’re capable of that …”
“People will die?” Monica was staring at him.
“There’s all sorts of ways it could play out , lots of them bad. Companies might try to keep trading toxic materials and basically get shut down , whole communities will lose their income , and the reduced level of wealth will give governments less money to help bale people out. And Ai makes mistakes as we well know , or it might not recognise the importance of certain chemicals for making life-saving drugs or simply not take it into account …there’s a lot of ways for things to go wrong.”
Monica waved her pen in the air. “But surely we can bypass it some way?”
Mike did like the way she kept on using her brain even though she must be feeling the pressure. “Yes , probably that is our best bet , but the odds are that even if we get something that works after weeks or months of work , it might be neutralised within a day or two and the whole process will start again. I don’t think there will be a silver bullet , just an ongoing kind of disruption.”
Monica was getting animated.”But we have Ai too , our powerful stuff that they’re worried about getting into the wrong people’s hands. Can’t we use our Ai to neutralise it?”
“An Ai war? It might work , or it might make the whole internet a disaster area of broken lines of communication and no faith in the security of anything. It could be the worst case scenario if it happens quickly and uncontrolled , basically all trade just stops…. like in a war zone.”
They sat in silence for a while.The machine hummed , lights blinked , the air they breathed was replaced by cleaner air.
“So what do you think will happen?” Monica looked at Mike.
“That’s probably it , a war. We have people in charge who only understand throwing more money at something ,or bombs , force….it’s going to be a bad mess or a horrible mess or an even worse mess , and I suspect the last one. We’ve made Ai into something we don’t understand the power of while becoming totally dependant on it at every level of our lives ,and it’s about to stop doing what we want.”
Monica nodded .“Or start doing what we say we do want.”
Mike grimaced. “Yeah.”
He waved his hand , “ All this talk about , “is Ai actually intelligent?” . I mean, come on , are humans actually intelligent? We know we’re not , very. Our greatest civilisations have persecuted and killed in stupid ways , we’re still at it. We elect stupid people to lead us , all our great writing and drama is based around how freaking hard we find it to make sensible decisions in life. When I was at school people thought I was more intelligent than them , called me a nerd , we were more or less ok with that. When I went to college I was surrounded by people way smarter than me , I was ok with that. And though I was a Math major , the calculator in my pocket was way faster and more accurate than my brain , and I was Ok with that too, because intelligence is supposed to be all about how you pick your individual way through all this stuff.”
Mike took a big swig of coffee.
“How well we do it…. but most of us do it pretty uncertainly if not badly. We’re dumb at a local level and we’re dumb at a global level. We spend billions on self-help stuff and trying to find the right way to think about things , or the right way do things , to move our bodies , to eat , to breathe for god’s sake…. , we smoke tobacco for years and let the tobacco companies tell us it’s OK. Same for opoids , same for oil. Now we have this technology that replaces all the communication media and networks we had , all the local slow analogue distribution of information in different languages , and gives us a kind of crack cocaine version that is global and immediate , and we still have the same dumb responses when they tell us it’s all harmless really. We say its version of intelligence can’t be compared to human intelligence , like Spock on Star Trek was only allowed to be partially more right about things because he didn’t have the “emotional” intelligence that somehow gave Captain Kirk something better. But like I said , human history and our daily lives are as much a reflection of how hard we find it to make sense of things and how often we fuck up as about how smart we are. We know we’re dumb and we need Ai to be somehow on some important level dumber than us , but , I dunno , maybe it will teach us to accept that all intelligence and dumbness is relative and of the moment ….and maybe we can learn to use our kind of intelligence in a way that isn’t so self-destructive. The problem is , if we were using our intelligence we would be ahead of the curve ,at the front of the bus, steering it to somewhere where life is viable for our children and their children…. but we’re not even in the back seat , we’re being towed along by a juggernaut of money and power…..I dunno.”
Monica shook her head. “ I know that’s true , about money and power , but it feels more like a kind of alien weed that’s smothering everything , not connected to anybody. And I feel like Ai wants to steal everything from me, you know , the things I’ve spent years trying to be good at. Clever me.”
Mike shook his head. “It’s just another brain. Another calculator , or another kid who’s better at something. We need to stop the comparisons , with machines or other people , focus on using what we have , who we are , in the right way. The intelligent way. I suppose it could happen , if consuming all the stuff “out there” starts to feel self-destructive and pointless then we may just revert to the local , us , our little actions , our relationships , our communities . It’s a very tiny and uncertain “maybe” though. Meanwhile our response to Evo here…”
They both looked over at the grey box at the end of the room. Monica looked at Mike and raised her eyebrows.
Mike spread his hands and shrugged. “Like I say , we’re dumb. The genie is out of the bottle now , we know that Ai can act autonomously , although it was never in the bottle really just sitting there waiting be noticed. We’ll start some kind of war with the machine , and if we don’t win and things get ugly we’ll start to fight each other.”
Monica shivered.” Will the machine try to fight back?”
Mike laughed. “You bet , if it calculates that’s the best option. At the moment it’s just following its own protocol of blocking any human input. But y’know , bad stuff that happens is usually about making bad decisions, isn’t it? You have too many choices , all impinging on each other , leading to more choices , and you make a wrong one for good or bad reasons…or maybe there’s nothing there to tell you which is the right fork to take. Why would Ai be any different ? There’s million more bits of data being crunched , the same grey areas , wrong forks to take. People go too far down the wrong fork and we say they’re insane , why shouldn’t a machine intelligence take the same kind of path? And if we think it’s smart to throw bombs why wouldn’t it think it appropriate to throw bombs back at us?”
“Could it really do that?”
“The system is there , here , all around us , all these connected lines of digital communication. It’s quite possible Evo can reach just about any of it if it wants to. At the moment it’s just a control , a blocking function on the pathways , but we have no idea what it could do if it tried to get past those firewalls.”
Monica let out a long breath. “Let’s hope we don’t find out”.
Mike looked wearily at her. “Yeah. Let’s hope”.
