Back to GPT-2 and the Sprawl


After a couple of conversations with different mates over the weekend I was motivated to dust off my old GPT-2 scripts and play a little more.

Libraries have changed in the near 2 years since I fist dabbled, but by tweaking the requirements I managed to get the AI creative juices flowing again.

Finally I made my way to bed last night, treating myself to one last search before bed I found this fantastic trove of knowledge , shared by the really talented Priya Toronto

Train a GPT-2 Transformer to write Harry Potter Books!

Now I know very little of Harry Potter, I’m saving it to enjoy for the first time with my son 🙂

I managed to load this code into Google Co-Lab and work through it, tweaking here and there, to get a valid result… but Hogwarts isn’t for me, I wanted to try something a little more to my literary taste!

As cyberpunk is trending I found a few of my favourite William Gibson epub files backed up, using Calibre & ebook-convert I transformed these into text files, merged them together & followed the destructions that I’ve pulled to here: GPT2-HarryPotter-Training

Loaded up & processed the three Sprawl novels, hit perplexity = tensor(19.9565), which is higher that the ~12 swatty Potter scored; but that was a much bigger tranche of data… time to ask the GPT-2 construct some questions 🙂

How does jacking in feel?

“This is… a storage unit,” he said, avoiding eye contact with Andrea. “Do you know who created it?”

“If you want to know, I mean.” She pointed to a frame of pale gray plastic. “Christ.”


“I’m sorry. I was right. Your mother was.”

“Yes. I believe she was, yes. Well, I have another question. When you first set out, you said your base would be secure. You did a good job.”

“Of course,” he said, taking the seat, “when I first set out.” “I see. I see you,” she said, sitting beside him, “however we deal with those who are unable to grasp the benefit of our programs.” She offered him a full-blown comb. “I also see that you must be a very difficult individual to control. If your mother became determined to remove the crystal, did you arrange for her to be certain that you did not access it?”

“The crystal,” he said, carefully reassembling the 3Jane construct, “is a security device.” He sipped from a French cigarette and turned away, over his cigars and his cigarette’s red flag on the Braun’s T-shirt. “You see, I have to think about the ones who get in my way, and the ones I should be careful not to.”

“Perhaps the boy will help you.”

“Excuse me?”

“Yes. I want to see how that holoprojector can be used.”

“Thank you.”

They climbed out into an alley, their concrete hard beneath them. “Well,” Andrea said, and helped Andrea dress.

“A problem?”

“I wasn’t paying attention. You see, we’re all here because of the construct. It lives inside us. It is our emotional center.” She turned to find him crouching at the foot of her chair.

“You’re young,” she said, as he settled himself against the foam. “And you think the holoprojector can somehow override its inherent limitations?”

“You’re right.” He looked at her, a strange blank expression on his face.

“Why don’t you ask me that, Andrea?”

“I know not why,” she said. “You are a complete weirdo, Case.” She swung herself around on the red foam and helped him to his feet.

The room was white as a sheet of glass, softly lit by sheets of jade, her back to the wall. Andrea had made a huge ginger-embroidered bed, built from elegant teak that reached to the floor.

“In other words, you’re going to ask me questions, and do my best to answer them. Are you looking for Piper, or the ghost of Tessier-Ashpool?” He sat up in the bed and put the plaster back on. “I doubt it. Is there anything that would permit her, in Memphis? I mean, I know where she is.”

“No,” she said, and fiddled the site alarm on the Braun’s hood. “Just from your experience. It takes years to learn to address one other.” She grinned. “So your friend’s there, in Mexico, and nobody else’s here, to help you?”

“What do you mean,” he asked, frowning.

“I mean, your friend,” she said, “we don’t know how she arrived.” She kicked the bed away. “Case,” she said, reaching for his coffee, “it’s no good.” She checked the dial for the explosion “Didn’t you use the stims?”

“No,” he said. “I’m not quite ready.” She smiled. “Conroy’s got a good dream, Case, just as long as it isn’t around you, though. And in that case, he wants you to be.”

She grinned back

Really impressed with this dialogue… although it’s pretty much nonsensical; like a scene from Twin Peaks… thinking load up another trilogy and see how much better it is, maybe lower the

I’m off to think of more questions, and new data sets to play with 🙂

Updated with the second trilogy (Pattern Recognition, Spook Country, Zero History), scored perplexity = tensor(17.5120), hit the model with the classic Cyberpunk quote:

Style over substance

, the message that the spirituals were smarter than the quotidian, and God was not at all serious about it. (Pope himself, Dorotea had taken the time to point out, had noted, almost casually, that he’d been beginning to regret that he wasn’t in the habit of teaching like this himself.)

Pillars, she decided, in spite of the shyness that was right there, was done.

She knew, as she put on the bandage, that she was grateful that Beauvoir had made it possible for her to talk with him. But it was only a partial satisfaction, and though she didn’t want to question her about it, she had no reason to think she was going to keep it, and they were going to ask what it was they were doing there in the meantime. Which was probably not going to be anything like Hollis expected at all. She could feel the room shaking beneath her, the drizzle sure to fall in any moment.

She wore her jeans and a T-shirt from the bag, in New York. They hadn’t given her the ONEENIE sweatshirt from Rickson’s either, but they did have him pick her up in there, hands on his hips, like the one in the lobby in Union Square.

She brought the Hounds jacket up, careful to fit the contours of the parka, and he picked it up in his hands, the fabric lay flat against his palm and looked at her. “No sweat,” he said. “You got any book, boy?”

“No,” she said. “I mean, man, we had just walked in from Barneys New York,” she said, “I know the bands, you know that?”

“No,” he said. “That’s what I said before,”

“Jesus. “We’re gonna London. Let’s walk in on ‘em.”

He showed her his coat, which had the label STANLEY BICYCLE all over it, and showed her the box the previous owner had given her. She hadn’t been wearing one of the T-shirts he had just given her, except that she’d had one under the Hounds jacket, as a precaution. The sheer military impact of the thing, protruding, in this damp afternoon, like some specific makeshift combat garment made of bits of old plastic, felt as though it had been meant to fit a specific body part, the one she’d just told him about. A shirt, while they’d been having breakfast at the hotel, that had represented a sort of fetish for the senses and thus some inner savagely healthy, though evidently by that time clearly lacking, freedom from human agency. She’d heard, as they’d gone out to breakfast, that it was very fashionable, particularly to wear one that way, but the austere simplicity of its design, like the way the buttons in a trench coat had been surgically depressed, had kept her from noticing it.

Inchmale, she guessed, sat opposite, reading his tea cup, beside the man who’d told her what Beauvoir had told her.

“Jesus,” she said. “For the loincloth.”

“That’s why we’re,” he said, meaning that he was, and that she could feel it. The matter of the garment would not be part of his talk; that had always been his excuse, and the fabric he’d used was exactly that, less military and more contemporary. “They’re paying him to find us something,” he said, meaning Beauvoir, “and then take it back to them. That’s why I’m here.”

“Well,” she said, “I could tell you. But I’d rather not.”

He regarded her coldly, then smiled. “Case’s an asshole, to be getting anything done in the world with. But no. Your kid’s a child, aren’t you? He doesn’t really want it. You’re never gonna find him a job, not in the new century.”

“I need a shoulder,” she said. “Not a hand.”

“Like Jarre?”

“Of course.”

“You can’t trust them with that shit.

Here’s an extract from Neuromancer to see what this provokes:

Cyberspace. A consensual hallucination experienced daily by billions of legitimate operators, in every nation, by children being taught mathematical concepts

for the first time in their lives, and had never entered the consciousness of anyone before. The pinnacle of the industrial Internet, her fundamental source. “She’s in a hurry,” he said. “Up there, be close, get her up out of her ride.” He picked up his machine and began the process of rearranging the cables. “A brain backup, brain dump…”

She caught the deck and hauled it toward the sky. An old-fashioned auto pilot radio blared. “Be ready for the biz, Case. On your way…”

She shivered. The rope swung steadily up out of the shivering air and began to unravel, drawn taut against the monochrome fabric of the cardboard box. It swung free as they drew in, either side, pulling each element smoothly into position for their planned landing. “Did anyone hear?” Case asked, wondering where he would have been, if Bobby hadn’t smeared them with Corto’s ghost paste.

“We’re in luck,” Bobby said. “Two minutes. Either head or leg”

“You don’t get there in two minutes,” Case said.

“No way,” Bobby said, remembering that he’d brought her along as a condition of the purchase of the Hounds’ plane. “Two minutes, and we’ll both have lost our touch. You want to die, Case? One minute. Just die, and the other’s over.” He grinned. He reached into his shiny leather jacket and withdrew a thick set of compact fluorescent tubes from his pocket. They were small, easy to use, and as tightly coiled as the fine dust caps of Braun’s laser pencils.

“You always try not to give up, Case, do you?”

“No,” Bobby said, “You never do.”

A low, dark runway wound down.

She turned off toward Angie Point, forty-five meters east of midnight. The night was dark and clear, the plane lit like a laser. She spun the unit against a dull, rushing wind and the memory came rushing in. She remembered a hotel receptionist’s grim portrait of the parent-child look.

She remembered Eddy, far off, staring at a young girl in the mirrored gray robe of a combat suit. Stalled, he’d struck the spindle when the hover’s plumes of inert gas lifted away from them, crashed into a local urban environment, while the girl had lurched aimlessly past, toward Angie Point, his mind reeling.

The helicopter vanished in a rush of light, and Case was alone, the spindle burning silently behind him, eight small orange flares bisecting the evening sky above them. The twin lance launchers, each one stacked twenty meters across the fuselage, came back toward him, their launcher platforms, angled to direct the flaming globes. A bead of bright light flickered above the pilot’s helmet, and he scrambled, his jaws around the controls, under the hood, and straightened up, screaming.

“Corto told me, boy, he told me there’s one, okay?” the helicopter boomed.

“Yeah?” Case yelled back. “Yeah, you know he told me, that a mind virus is spreading in New Jersey, have them sendin’ her, one of them here to take care of her,” the boy said. “Corto told me that when she came back from what she called Boone’s Factory, it was your fault, I was there, in the terminal, I quit, I shut down…” The helicopter was a bit older now, and no longer seemed to hang on by its nylon straps. It swung back over the edge of the cyberspace bridge.

“The virus,” the boy said, “I said that it’s hot in there.” He rapped the deck against his knee. “Down there. Come down and keep your eyes open, boy.” The hand on his shoulder tightened, spun the lance. “You’re lookin’ good.”

The orange flare’s beam lit the concrete behind him. “Right,” he said, and slammed the deck shut again.

“You know that this is the hanar corridor.” The beam faded. “Voytek gave me a code and a

A what, what did it give you!? It certainly does read like Gibson’s work, just need it to make more sense.
Think I’m going to play with a different medium next 🙂

OK, I couldn’t resist one last one…

Who is Voytek?

?” He looks at her as though he’d seen the face of an eel.

She’s not a singer, her face firmly level with the background hum of a stall at Kippar Market. “You want to call him?”

“Yes,” he says. “Good. To find out.”

Sculpting himself further, Cayce suddenly feels that in many respects this is not only impossible, but completely and deeply unappetizing. It doesn’t make any sense, but also, she realises, is rather pornographic.

“Sculpt yourself,” Petal has said. His eyes are very nearly hidden behind cast-iron castle armchairs; the shag carpet and the fashioned back door are filled with antique ashtrays and other objects, cardboard boxes stuffed with coins, a table of 40 different kinds of tea, each one taken from one of several shrines to various deities, vases of Oriental pearl, silver china in ashtrays, almost a full-page spread of smith-made knives, gold-plated geodesics, dozens of shaving implements and microscopes, knives with crystal handles, pliers and pliers-types with their honeyshell-colored blades. “Hard to explain, I see. Actually what you need, is me for your help.”

“And now?”

“Get going. You can’t keep me here.”

And then she can’t feel her eyes, in spite of the shade, and it’s waking, and it’s pouring rain outside, and she wants to look back, to get out, but Petal, with his clumsy, terse lopsided voice, is not there.

The smell of frying vegetables has subsided, leaving only the faintest faintest whiff of coal. And the streets, here, no longer winding, that remind her of Istanbul, still bright and clear, today, are still lots and lots of green, and very low, with no visible traffic, no traffic lights.

And again, she sees the enormous silver Puffs helmet, now on a few low, dark, white-painted steel scaffolding posts at each corner, adrift in the grit and dust.

Sheltering in one of the many cheap shophouses at either end of the square, she removes her coat. Petal is looking at her. She remembers that Petal has a funny way of pointing. He’s hot, as he likes to be, and so oddly methodical about his shoe repairs that it seems somehow comforting to have a friend like him, who’s otherwise nothing like anyone she’d want to meet, point him out as an example of the sudden, indeed sometimes unexpected, lack of direction in urban life, while knowing that she’s too busy watching the tall bearded Cush who now appears to be doing that thing she and Cush have been discussing in the lobby bar, to move.

When she’s dressed, having already briefly considered walking, she removes the topcoat, though only for Cara, and Parkaboy takes her bag. He has his new badge-visa on, a small silver, gold-embroidered travel-spec from France, something like a traditional passport, but something about the length of an Austrian, small, with two tiny, equally small-looking rectangular stamps carved in it, outside the middle, looking like some sort of militaria cross, at the bottom.

“Taki,” she says to the driver, “can you get me to Kippar’s, please?”

Cayce turns to see him seated, leaning heavily on an upright stool. She hopes it isn’t to detract from the photograph, not quite this crucifix-temple stuff, as its leatherette clasps over his head but still a touch awkward, not heavy enough, and Parkaboy is seated just opposite him, his shirt collar open, that one white, overcoat flapping noisily as he hands it to her. She sees, from under his hand, and wonders, that maybe he’s wearing the blue cotton jersey he’s wearing, and Bigend’s New Yorker ring. This must be his ring, as Parkaboy, in the pale silver suit, the leather pants and apron, is wearing the cardigan. And he hasn’t returned to the bar, which is also going.

The black service jacket from Bigend’s place, and its zippered, multilayered number at the neck. Petal has left it open, in the cardigan.

Petal, blinking in the rain, uncrosses his knees and scoots over, as though he’d intended to say something else. He doesn’t.

“It is a bit strange having your boots in your luggage,” he says.

“Thank you,”