(* * LANGUAGE : ANS Forth * PROJECT : Forth Environments * DESCRIPTION : simple neural net (no backpropagation) * CATEGORY : Example * AUTHOR : Marcel Hendrix, June 1991, MHX * LAST CHANGE : October 13, 1991, Marcel Hendrix *) (* -------------------------------------------------- *) (* N E U R A L N E T D E M O *) (* This listing date : MAY 12th 1990 *) (* Public Domain *) (* This software may be freely copied. *) (* *) (* FC: Networks of Neurons Robert E. La Quey FORML 87 *) (* LC: Scott Squires uploaded it to GEnie *) (* LC: Phil Burk 7/14/89 Some modifications *) (* *) (* Adjusted for F-4TH May 1990, MH *) (* Fancied up the display to '90 standards.. *) (* Bug: Runs so fast as to be unusable. *) (* This file has NOTHING to do with backpropagation! *) (* *) (* Adjusted for tForth June 1991, MH *) (* -------------------------------------------------- *) NEEDS -miscutil NEEDS -terminal REVISION -neunet "ÄÄÄ Neural Net demo Version 1.02 ÄÄÄ" PRIVATES DOC NeuralNets (* ( Networks of Neurons Robert E. La Quey FORML 87 ) The following is source code presented by Robert La Quey at the FORML conference in November, 1987. This was released into the public domain so that people could become familar with neural nets and how they could be done. For more info see Mr. La Quey's paper when the FORML proceedings become available. -Scott Squires GEnie S.W.SQUIRES ( Comment from the author [?]) Because the weights are equal for each input the problem can be scaled so that no multiplication is required. Thus the ON value is given by the max number allowed divided by I/L. For purposes of this simulation I have built a byte oriented simulator with I/L = 8 so that ON = 32 ( 256/8 ). I suspect that the choice of equal weights does not involve any true loss of generality. By connecting an output to several computing elements we can obtain results equivalent to the use of weighted inputs. The connection matrix contains an element# for each input of each element. I use byte cells and hence have allowed for 256 elements. The connection matrix uses #L I/L * = 2048 bytes. One quickly becomes impressed with how slow the IBM PC is. This type of simulation is computationally intensive. There are #L I/L * (here 2048) operations per step, i.e. essentially one operation per connection, in the inner loop. Thus one quickly loses interactivity and those of us with short spans of attention lose interest. The Chuck Chip is fast enough to allow useful experiments. ( End author ) MOD: Phil Burk 7/14/89 Added DECIMAL at beginning some more comments, ANEW, .NET.CONTROL added to .NET fixed stack problem in GreyMatter. ------- Explanation of the Code ------ #L is the number of computing elements (neurons) in the network. Each computing element has I/L inputs/element. The output of the computing element is scaled so the product I/L NONN * = max range = 256 here. THOLD is the threshold number of inputs. When more than THOLD inputs are = NONN the computing element fires ... and sets its output to = NONN. The Connection Matrix describes the network. For each input of each computing element of the Connection Matrix contains the number of the computing element connected to that input. The buffers BUF0 and BUF1 contain the current and previous values of the computing element outputs. At each step in the computation the current output is used as input, the previous output is overwritten, and then the buffers are swapped in preparation for the NEXT step. QUIET puts zeros (NOFF) into the holding registers. This corresponds to a quiescent network with no initial thoughts. Perhaps this is an example of the ZEN notion of NO MIND. Exercise for the student: What is the sound of one neuron clapping? We will need to CONNECT elements to specific inputs. Also to DISCONNECT them. (Note DISCONNECT DISCONNECT does nothing) Given an element# and an input# CONNECTION returns the element# that is connected to that specific input. NEURUN is the basic simulator. It simply evaluates the total input weight by summing the inputs and compares it with the threshold. If the threshold is exceeded NON is put into the output register else NOFF is placed in the output register. NEURUNS just does it for all of the neurons except for 0 thru 7 which are reserved to input data from the external world. THINK begins from a state of NO MIND and computes. IS-NET-INPUT ( ch -- ) maps an 8 bit item on the stack from the world external to the Neural Network onto the holding registers for the computing elements numbered from 0 to 7. Thus as far as the network is concerned the outside world just looks like the output of neurons 0 thru 7. These elements do not change state as the network computes but simply hold the input character. NET-OUTPUT-IS ( -- ch ) maps the output of the last 8 computing elements (neurons) onto the stack. This screen shows one of an infinity of ways of specifying a Connection Matrix. For this particular choice: a) inputs 0 thru 3 are driven by randomly chosen elements from the outside world. b) inputs 4 thru 7 are driven by randomly chosen elements from the internal world by the network. Each element is connected to the outside world (like a retina?) and also to the internal world of the network. The I/O words provide a simple display system which should be portable to a wide variety of computers. ( General READ.ME info stuff ) Neural Net Simulator Original code by Robert E. La Quey This README file and some minor mods by Phil Burk Modifications to this file (code and doc) Marcel Hendrix. JFORTH specifics were removed. Some F-4TH and tForth specifics may have crept in (VALUE ENDIF =: 2^x REVISION). This code is now once more readable on any machine that can handle ASCII text files. But be warned: this is a rather dull program if you have a fast machine. This code is Public Domain and may be freely redistributed. What is a Neural Net? A neural net is a crude simulation of the brain. Neurons can be connected to one another. Each neuron can receive input from several neurons and give output to several neurons. If enough inputs are on, the neuron will fire. This can, in turn, cause other neurons to fire. Patterns can propagate through a neural net in complex ways. Neural Nets are a hot topic in computer science these days. They have been able to "learn" how to do tasks by observing humans do them. This learning constitutes a form of self-programming that will someday eliminate the need for us programmers. Hey, wait a minute! Nah, it could never happen. :-) This a very crude description of neural nets by someone who knows almost nothing about them (PLB). If you are more knowledgeable then I, please, offer a fancier neural net program and description. There is an article on Neural Nets in Scientific American, Dec. 1987. What Does This program Do? This program sets up an array of neurons and connects them up randomly. It then calculates a random input. The input is the top row of 1's and 0's. The neuron array is the big block of 1's and 0's. The input propagates through the net and appears on the outputs, the bottom row of 1's and 0's. You can pick a new random input by hitting 'R' and a new set of random connections by hitting 'C'. Do this, then hit the space bar several times and watch how the output changes. I suggest you peruse the source code to see how this happens. What Do I Do? To run the program, fire up your favorite Forth interpreter (tForth F-4TH, FysForth or KFX-Forth will do) and type: INCLUDE NEUNET.TXT DEMO This ReadMe file by Phil Burk. *) ENDDOC DECIMAL 64 =: #L PRIVATE -- # of elements 8 =: I/L PRIVATE -- Input/element 256 I/L / =: NON PRIVATE -- Neuron has fired 0 =: NOFF PRIVATE -- Neuron has not fired 3 =: THOLD PRIVATE -- # inputs required to fire THOLD NON * =: GAMMA PRIVATE 0 VALUE STEP# PRIVATE -- iteration counter CREATE >CM PRIVATE -- Connection Matrix #L I/L * ALLOT -- fill with values yerself! : CM ( el# in# -- addr ) \ offset + addr in Connection Matrix array SWAP I/L * + >CM + \ last plus was missing in the original! ; PRIVATE 0 VALUE >OLD PRIVATE -- holds addr of current "input" buffer 0 VALUE >NEW PRIVATE -- holds addr of current "output" buffer CREATE BUF0 #L ALLOT PRIVATE -- create two buffers which will swap ... CREATE BUF1 #L ALLOT PRIVATE -- ... functions every interation : START ( -- ) \ init counters & arrays 0TO STEP# \ start counter BUF0 TO >OLD \ assign input buffer BUF1 TO >NEW ; PRIVATE \ assign output buffer START \ init this part of system : NEXT ( -- ) \ Swap I/O Buffers 1 +TO STEP# \ increment counter >OLD \ get addr of previous input buffer >NEW \ " " " " output " TO >OLD \ and reverse... TO >NEW ; PRIVATE \ ... the buffer assignments : INPUT ( el# -- addr ) \ add array address to offset >OLD + ; PRIVATE : OUTPUT ( el# -- addr ) \ add array address to offset >NEW + ; PRIVATE : QUIET ( ---) \ clear the working bytes in both buffers 8 INPUT #L 8 - ERASE \ erase 56 bytes of "input" buffer 8 OUTPUT #L 8 - ERASE \ erase 56 bytes of "output" buffer ; PRIVATE : CONNECT ( el# el# in# -- ) \ fill in a byte entry in Connect Matrix CM C! ; PRIVATE : DISCONNECT ( el# in# -- ) \ negate a byte entry in Connect Matrix CM DUP C@ NEGATE SWAP C! ; PRIVATE : CONNECTION ( el# in# -- el# ) \ inspect a byte entry in Connect Matrix CM C@ ; PRIVATE DEFER DISPLAY PRIVATE : NEURUN ( el# -- ) \ Runs a single neuron 0 SWAP \ dummy entry, then bring element number to top I/L 0 \ do for input-per-element number of iterations ?DO DUP \ the element number ( --- 0 el# el# ) I CONNECTION \ get matrix entry ( --- 0 el# entry ) INPUT \ use as index to the current "input" buff \ ( - 0 el# ad) C@ \ fetch value there ( --- 0 el# value) ROT + SWAP \ ( --- value el#) LOOP SWAP \ cumulative value to top ( --- el# value ) GAMMA < \ within threshold ? ( --- el# t|f ) IF NON ELSE NOFF ENDIF SWAP \ ( val el# --- ) OUTPUT C! ; PRIVATE \ put in current output buffer ( --- ) : NEURUNS \ Runs the Neural Network #L 8 DO I NEURUN LOOP ; PRIVATE : THINK \ <> --- <> QUIET \ clear inp/output buffers BEGIN NEURUNS DISPLAY NEXT AGAIN ; PRIVATE : IS-NET-INPUT ( ch -- ) 8 0 DO DUP 7 I - 2^x AND IF NON ELSE NOFF ENDIF DUP I INPUT C! I OUTPUT C! LOOP DROP ; PRIVATE : NET-OUTPUT-IS ( -- ch ) 0 8 0 DO #L 8 - I + OUTPUT C@ NON = IF 7 I - 2^x + ENDIF LOOP ; PRIVATE : GreyMatter \ connect neurons for demo. \ ( GM for Grey Matter ) #L 8 - 8 \ Connect internal neurons except for \ output. Connect 1/2 of the inputs of DO \ each neural element to one of the 8 I I/L \ inputs from the outside world. 2/ 0 DO \ ( -- K , i from outer loop ) DUP 8 CHOOSE \ ( -- K K rand ) SWAP I \ ( -- K rand K I ) CONNECT LOOP \ Connect the other half of the inputs \ of each neural element to one of the \ other previous neural elements. I/L I/L 2/ DO DUP DUP CHOOSE SWAP I CONNECT LOOP DROP LOOP \ Connect output neurons #L DUP 8 - DO I I/L 0 DO DUP #L CHOOSE SWAP I CONNECT LOOP DROP LOOP ; PRIVATE (* Neural Network Portable Character Display *) : ?CR 16 MOD 0= IF CR \ --- <> ENDIF ; PRIVATE : .ON NON = IF '1' \ --- <> ELSE '0' ENDIF EMIT ; PRIVATE : .STATE DO I ?CR \ --- <> I OUTPUT C@ .ON LOOP ; PRIVATE : .NET-INPUT CR ." Input: " 8 0 .STATE ; PRIVATE : .NET-OUTPUT CR ." Output:" CR #L DUP 8 - .STATE ; PRIVATE : .NETWORK CR #L 0 .STATE ; PRIVATE : .CON CR DUP . CR I/L \ --- <> 0 DO DUP I CM C@ 4 .R SPACE LOOP DROP ; PRIVATE : .CONS CR ." Connections " #L 0 DO I .CON LOOP ; PRIVATE : .NET.CONTROL EKEY? 0= IF 0 L/SCR 5 - AT-XY \ allow interactive control ." Hit '?' for help.." HOME EXIT ENDIF CURON EKEY >UPC CR CASE 'R' OF ?AT 256 CHOOSE DUP IS-NET-INPUT HOME ." Input is (hex) " H. AT-XY ENDOF 'C' OF GreyMatter ENDOF '?' OF ." 'Q' or 'ESC' to quit" CR ." 'R' for new random input" CR ." 'C' to reconnect neurons" CR ENDOF 'Q' OF EOS QUIT ENDOF ESC OF EOS QUIT ENDOF ENDCASE CUROFF HOME ; PRIVATE : .NET .NET.CONTROL \ display network status as 1s and 0s. CR STEP# . \ ( added by PLB ) CR .NET-INPUT CR .NETWORK CR .NET-OUTPUT ; PRIVATE ' .NET IS DISPLAY : DEMO CLS CUROFF \ demonstrate neural network START \ initialize neural system ['] .NET IS DISPLAY \ make DISPLAY use the .NET definition GreyMatter 256 CHOOSE DUP ." Input is (hex) " H. CR IS-NET-INPUT THINK ; \ break with 'q' or ESC :ABOUT CR ." USAGE: type 'DEMO' " CR ." (break by typing 'Q')" ; DEPRIVE .ABOUT -neunet (* End of Neural Source *)