In Awe

How dare you speak to me like that!” the drone bellowed. “I’ll have you know I am an Accredited Free Construct, certified sentient under the Free Will Acts by the Greater Vavatch United Moral Standards Administration and with full citizenship of the Vavatch Heterocracy. I am near to paying off my Incurred Generation Debt, when I’ll be free to do exactly what I like, and have already been accepted for a degree course in applied paratheology at the University of—”

“Will you shut your goddamn… speaker and listen?” Horza shouted, breaking into the machine’s breathless monologue.

I am reading Consider Phlebas, the first book in Iain M. Banks’ enormous Culture series. He wrote it in 1987, and it contains some of the most remarkable robotic characters I’ve ever encountered in fiction. These passages, stripped of other context so I hope no spoilers, but the characterization here is so remarkable I feel the need to highlight it as an inspiration for later.

“Oh good grief, man!” the drone exclaimed. “What do you want? A robot?” Its voice sneered. “I don’t have an Off button on my reasoning functions; I can’t choose not to have free will. I could quite easily swear to obey all orders regardless of the consequences; I could vow to sacrifice my life for you if you asked me to; but I’d be lying, so that I could live.

“I swear to be as obedient and faithful as any of your human crew… in fact as the most obedient and faithful of them. For pity’s sake, man, in the name of all reason, what more can you ask?” …

“It might,” the drone said, rising from the table to the level of Horza’s eyes, “seem amusing to you, but it matters to me. I am not just a computer, I am a drone. I am conscious and I have an individual identity. Therefore I have a name.”

“I told you I’d use it,” Horza said.

Anyway, that’s from the middle somewhere. But it is such a superb look at how machines might behave.