Re: Wordsearch:AI Insanity

On Sat, 24 Mar 2007 08:36:00 GMT+1, Tina_Hall@xxxxxxxxxxx (Tina Hall)
wrote:

James Eades <jeeades@xxxxxxxxxx> wrote:

<snipping a bunch of stuff I'd love to follow, but don't have time...
sigh.>

Okay, back to the subject - bios... bios... I hadn't thought that much
about bios. Actually, I seemed to have assumed that the AI was the
bios, what with being hard-coded and all.

Makes sense. There is still information that can be changed and it
remembers.

The computer includes both the hardwired AI shell and transient data
storage for short-term memory work, as well as the data handling unit
with the 'handwavium' data storage.


The AI and the Mission (protect the... what DO you call the guy being
protected?

Client? Charge?


Charge, maybe.

It'll come to me. Don't help me, I'll get it any second....
....or not. <g>

I'll keep trying.

<more snip>


What state would the AI be in, when it studies the new instructions
and finds that they override all previous commands? At this stage of
the story (my own experiences, heaps of computer lore, mounds of
speculation, a modicum of politics and a smattering of child
psychology), the AI is 'smart' but not 'aware[*].' What does it do
when it encounters a command it should not obey?

Stop with an error message.

My thoughts (ignore if it violates your story)...

The thing is, with just software, there's no 'it should not obey', it's
an undefined command or something, so what happens depends entirely on
the software. If I press a key that has no function, nothing happens. If
I type "get lost [enter]" at the DOS prompt, I get a funny message (I
configured it to throw out one of a number of daft comments at an
unknown command), and nothing else. If I fiddle around with Quick Basic,
on the other hand, things happen in a way I sometimes never figure out,
and sometimes it breaks with an error message. You could have "adress
not found" or "division by zero" or whatever. I've seen enough error
messages that just throw cryptic numbers at me, playing Nethack,
Slash'EM, and Crawl. They exit the game and nothing but fixing the
sourcecode will get the game past that stage.

Did you ever work with the Commodore Pet? They had their keyboards
set up to enter graphic symbols (which were character sets) as well as
alphanumeric characters. It was possible to enter a machine code
command into the computer using graphic symbols.

In the early 1980s the public library in Austin, Texas, had Commodores
we could reserve and play with. I found that my daughter had a talent
for hitting random keys and sending the screen into some strange,
bizarre frenzy, sometimes locking the cursor halfway across the
screen, sometimes converted typed letters into gibberish. Had to
reboot the machine to get it back to normal, and sometimes even that
didn't help.

(Perhaps that was the story seed, a computer driven crazy not by the
software but by unexpected responses from an input device the
programmer didn't anticipate.)


Then there's ordinary bugs that don't stop the software, where a program
does strictly what it's told, but not what is intended. In Ultima VII,
objects (keys, tables, walls, roofs,...) disappeared from the landscape
when you were carrying too many objects around with you. In Railroad
Tycoon, you could get ~32k dollar in the red, and it would switch to
black, positive cash. In Civilization, you could sentry a unit in a
city, then reactivate it with full movement points restored. In Nethack
there was a version where you could teleport with the Amulet of Yendor
if your mount carried it. The list is endless.

There can't be a conflict of two options, it's either 0 or 1. A criteria
is met and it does A, or a criteria is not met and it does not do A and
waits until it is met|does not do A and checks for a different
criteria|does B. Whichever it is, it's tested one at a time.

Or, there is option C, where it cannot do either. It hangs. Like a
computer with device drivers misaligned or installed incorrectly .


I speculate that it would simply shut down, go catatonic from
overload, thus preventing itself from taking action.

If that's the way it's programmed, then that's what it should do. "If
target = charge then goto end" :)


(The college professor who taught me structured programming would roll
over in his tenure if he heard that 'goto' command. <g>)
I assume that the guy who programmed the AI would have included
filtering algorithms to take care of an erroneous command like that.
Being a naive nerd, though, he would not have included a double check
to ensure that the command was not reloaded despite having been
banned. (I'm playing fast and loose with my AI languages, here.)

However, I'm concerned about how to describe it if if it were to wake
up and resume operation, carrying out its new commands, waiting until
the right moment to attack its prey.

Why should it wake up?

Does it enter a loop or something, that's basically just a counter? No
conditions are changing, after all.

Not to go into too much detail and dull my anticipation for the
writing, it does enter a loop, in a strictly maintenance mode, to keep
only essential activity going.


This is the state of mind that I am trying to find a word/phrase to
describe - Hacked, subverted, corrupted, gone rogue, buggy,
virus-infected, abherent, poisoned....

I'm still in favour of 'warped'. :)

:)

I find myself reaching for human psychological terms - neurotic
because the AI is in distress, schizoid because it becomes isolated
and withdrawn, obsessive-compulsive.... Might as well throw in
multiple personality disorder, while I'm at it, as well as 'under the
sway of a hidden compulsion.'

Hm. Brainwashed? RAMwashed?

Bravely exploring new territory.... Surely someone has used that,
somewhere. RAMwashed seems too logical to have been ignored. If not,
then it's yours to cherish. ;-)


[*] In the course of the (projected) storyline, the AI would gain more
self-awareness, but not in the traditional direction.

So it's programmed to alter its own programming depending on input?

I would expect that to be the definitive requirement for an
intelligent system. The only thing that can't change (or at least so
the programmer thought) is that portion of read-only-memory housing
the prime imperative. (or 'prime directive', but ST has used
'directive' so much they have a prior claim on it) What the AI would
alter would be part of the interface between it and the outside world
- the very interface it has decided that it can no longer trust.


That would definitely mean an error message when it finds it can't alter
the ROM. And I suppose you don't want it to delete the bad guys'
programming.

The flaw in the perfect plan, of combining cutting edge cybernetics
with cutting edge programming, is that the programming may have to be
perfected before the hardware, and thus tested out with older
equipment. In this case, the hacked data/code was inserted before
final testing and the AI sent the error message to the old display,
the one which had been superceded by the fancy new handwavium data
handling unit.

What effect should the data labeled 'unused' have, anyway? (I read your
other replies in this thread as they got here.)

(I'm having to do some fancy tap-dancing/noodling here to explain my
way out of plot holes. Amazing how much you can learn about a story
when you're trying to keep from looking like a fool. Thanks. <g>)

Still, I know how to cover it. I just hadn't thought it through far
enough, before.

Short answer: The AI, if it ever re-awoke, would attempt to use the
storage address, discover that it had already been 'burned.' One of
two things might then happen: 1) the 'unused' label could be reset to
'deleted' in the computer's rewriteable register area (a couple of
gigabytes) and the data effectively discarded, or 2) the corrupted
data could be read and acted upon, at which time we would have a rogue
robot seeking to carry out the hacked mission.

(Which event we choose would depend on how much suspense we think the
reader could endure. IMO, all we can possbly induce. <g>)


Anyway, you don't need to guess what it would do, just have your
original programmer enter a few failsafes that prompt whatever behaviour
you want when they're triggered (don't let him miss too many obvious
ones, though). Good programmers do that, even when they think the
condition will never occur.

Beta tests are for optomists. ;)
__
JamesE
.

Relevant Pages

  • Re: Great SWT Program
    ... And, of course, none of the nasty things that you have said or implied ... such rot and squeezing out Java programming related traffic with emacs ... I didn't claim that they could not learn a load of programming skills ... bit is the entire command but is only part of the full command line, ...
    (comp.lang.java.programmer)
  • Re: Great SWT Program
    ... such rot and squeezing out Java programming related traffic with emacs ... I didn't claim that they could not learn a load of programming skills ... CLI skills are hardly "much of the skillset of programming" - but I ... bit is the entire command but is only part of the full command line, ...
    (comp.lang.java.programmer)
  • Re: Overflowing a Long
    ... Maybe you had to "grow up" on a command line to appreciate the clarity ... strictly in terms of 3GLs or even machine language. ... I would guess that you probably started programming even earlier ... and thus you're more deeply entrenched in the programming style ...
    (microsoft.public.vb.general.discussion)
  • Re: dynamic menus
    ... I'm new at Windows ... programming, so I keep getting tripped up by little things like this. ... > Most likely your objectMenu is going out of scope and being destroyed. ... > Each command ID generated by the menu items must be unique. ...
    (microsoft.public.vc.mfc)
  • Re: [SLE] C Programming in SuSE
    ... C Programming in SuSE ... > and it gives me a message: bash first no such command. ... > and it gives me the same error message. ...
    (SuSE)