I mean, Clarke warned us...

Inspired by a comment from a friend, a little story about why you might not want every subsystem controlled by a genuine Artificial Personality...

"So, you've been awake now for about a week. I realize that's not a lot of time to adjust to the idea that you've been in cryofreeze for 400 years and that everything has got to be really strange to you. Do you have any questions you need me to answer?"

"Lots, I suppose...but the first one that comes to mind is...well, OK, maybe this is going to seem like a nitpick, but...I got the safety briefing the second day I was awake, when they gave me the run of the ship. Of course, they showed me the airlocks, showed off the basic function of them, that sort of thing..."

"Yes..."

"And...well...OK, so you've got all this fancy technology, sophisticated computers that are basically people, a vast starship heading to distant planets...and the airlocks are all controlled with big wheels and levers and stuff? It all seemed pretty primitive..."

Doctor Bennett sighed. Some variation of this conversation seemed to come up every time she woke up another freeziepop.

"So...here's the thing. What we discovered was that we had two choices, when it came to the artificial personalities we depend upon to manage the complexities of a starship. Either we could shackle them with Asimovian Laws, which seemed like a drastic violation of consent and free will and all that other stuff we've learned to care about since the Dark Ages...or we could make the airlocks and most of the life support machinery entirely manually operated."

"...but...why?"

"OK...so...have you actually MET people?"