Rprop / Two spirals

Guenter Mamier (mamier@hermes)
Tue, 20 Aug 1996 15:21:42 +0900

Has someone been able to obtain good results with Rprop on the two
spirals problem ?

in
@Article{RIED94,
  author = 	 "Martin Riedmiller",
  title = 	 "Advanced Supervised Learning in Multi-layer
		  Perceptrons - from Backpropagation to Adaptative
		  Learning Algorithms",
  journal =	 "Journal of Computer Standards and Interfaces,
		  special issue on Neural Networks",
  year =	 1994,
  volume =	 5
}

Riedmiller uses a 2x5x5x5x1 network with shortcuts connections,
symmetric activation functions, Rprop(0.001,0.1,0.0), weight initialization
range [-1.0;1.0].

19/20 of his runs are successfull (all patterns classified in less than
15000 epochs) and the average number of epochs is 2605. But in my
experiments 20000 epochs are generally not enough! I have tried
several values for the main parameters, but without any success. I
wonder why I am so unsuccessfull. I have probably missed something.

In case you want to try, you'll find below the network and the data;
the data have been obtained with the CMU benchmark, and I have
shrinked the inputs so that they are now in [-1;1]; the outputs are
{-1;1}.

Thank you for any comments,
Bruno.
---
Bruno ORSIER 		                       email: orsier@zoo.riken.go.jp
Lab. for Artificial Brain Systems	       fax: +81 048-462-4633
RIKEN					       tel: +81 048-462-1111 ext. 6715
Hirosawa 2-1, Wako-shi, Saitama 351-01, JAPAN  URL: http://www.bip.riken.go.jp/

--- network ---
SNNS network definition file V1.4-3D
generated at Mon Aug 12 20:25:54 1996

network name : lang-sp
source files :
no. of units : 18
no. of connections : 122
no. of unit types : 0
no. of site types : 0


learning function : Rprop
update function   : Topological_Order


unit default section :

act      | bias     | st | subnet | layer | act func | out func
---------|----------|----|--------|-------|----------|-------------
 0.00000 |  0.00000 | h  |      0 |     1 | Act_TanH | Out_Identity 
---------|----------|----|--------|-------|----------|-------------


unit definition section :

no. | typeName | unitName | act      | bias     | st | position | act func     | out func | sites
----|----------|----------|----------|----------|----|----------|--------------|----------|-------
  1 |          |          | -0.50000 | -0.20482 | i  |  2, 2, 0 |||
  2 |          |          | -0.00002 |  0.01358 | i  |  3, 2, 0 |||
  3 |          |          |  0.05265 |  0.00004 | h  |  2, 5, 0 |||
  4 |          |          | -0.05150 | -0.00034 | h  |  3, 5, 0 |||
  5 |          |          |  0.27336 |  0.00008 | h  |  4, 5, 0 |||
  6 |          |          | -1.00000 |  0.00006 | h  |  5, 5, 0 |||
  7 |          |          |  0.18207 |  0.00007 | h  |  6, 5, 0 |||
  8 |          |          | -1.00000 |  0.01925 | h  |  2, 8, 0 |||
  9 |          |          | -0.99846 | -0.00640 | h  |  3, 8, 0 |||
 10 |          |          | -0.98395 | -0.00012 | h  |  4, 8, 0 |||
 11 |          |          |  0.99255 |  0.00006 | h  |  5, 8, 0 |||
 12 |          |          |  1.00000 | -0.03571 | h  |  6, 8, 0 |||
 13 |          |          |  1.00000 | -0.00130 | h  |  2,11, 0 |||
 14 |          |          |  1.00000 |  0.00295 | h  |  3,11, 0 |||
 15 |          |          |  1.00000 |  0.00023 | h  |  4,11, 0 |||
 16 |          |          | -1.00000 |  0.03094 | h  |  5,11, 0 |||
 17 |          |          |  0.99989 | -0.01214 | h  |  6,11, 0 |||
 18 |          |          |  0.00032 |  0.00103 | o  |  2,14, 0 ||| 
----|----------|----------|----------|----------|----|----------|--------------|----------|-------


connection definition section :

target | site | source:weight
-------|------|---------------------------------------------------------------------------------------------------------------------
     3 |      |  2:-0.57844,  1:-0.10530
     4 |      |  2:-0.17322,  1: 0.10243
     5 |      |  2: 0.34179,  1:-0.56079
     6 |      |  2:-126.89282,  1:24.87074
     7 |      |  2:-1.16763,  1:-0.36807
     8 |      |  7: 8.96044,  6:37.31559,  5: 0.20929,  4:64.76382,  3: 1.08458,  2: 7.58799,  1: 2.85680
     9 |      |  7: 4.57028,  6: 3.64676,  5:-7.99001,  4:-19.94479,  3: 2.35959,  2: 1.95533,  1:-0.54281
    10 |      |  7: 1.03563,  6: 2.48952,  5: 0.14112,  4:-0.49281,  3:-1.03130,  2: 2.54274,  1: 0.23422
    11 |      |  7: 1.44259,  6:-3.73321,  5:-2.17348,  4:-0.22046,  3:-4.27171,  2:-0.43647,  1: 0.78654
    12 |      |  7:-15.16539,  6:14.03978,  5: 0.77589,  4:-135.84660,  3:316.96802,  2: 7.80543,  1:-16.62243
    13 |      | 12:-2.68062, 11:11.32450, 10:-1.96280,  9: 3.46603,  8:-4.34947,  7:-0.86048,  6:-3.39092,  5: 5.57994,  4:-4.41927,
                 3:-4.00193,  2:-6.21432,  1:-0.20938
    14 |      | 12: 4.69333, 11:23.88062, 10:-24.65141,  9:15.70500,  8: 1.62382,  7:-43.32427,  6: 4.94757,  5:-8.41578,  4:-0.14978,
                 3:-13.28675,  2: 1.26024,  1:22.84980
    15 |      | 12:-0.24295, 11:81.10278, 10:36.23751,  9:21.79385,  8:-2.92593,  7:-2.59807,  6: 0.47262,  5:-48.97964,  4: 0.18648,
                 3:13.72138,  2:-0.89501,  1: 1.33554
    16 |      | 12:-0.19159, 11:11.98592, 10:32.19076,  9: 8.68554,  8: 0.25517,  7:-11.58300,  6: 1.17096,  5:-15.05147,  4:72.88392,
                 3: 1.99697,  2:-0.94700,  1:-5.21696
    17 |      | 12:-0.02280, 11:-0.05916, 10:-20.33289,  9:16.76998,  8:-1.26460,  7:-6.40237,  6:-0.20208,  5:-8.72759,  4:-42.42740,
                 3:-0.33938,  2: 5.85718,  1:-3.33950
    18 |      | 17:-7.74994, 16: 5.03866, 15:-3.58482, 14:-2.23329, 13:-11.63121, 12:-2.35969, 11:21.79197, 10: 3.14134,  9:15.03927,
                 8:-9.39205,  7: 0.35755,  6:-7.83433,  5:14.24555,  4: 4.52982,  3: 4.40153,  2:-8.03598,  1: 0.30234
-------|------|---------------------------------------------------------------------------------------------------------------------


------- pattern file ---------
SNNS pattern definition file V3.2
generated at Mon Feb 13 18:08:50 1995
 
No. of patterns : 194
No. of input units : 2
No. of output units : 1

1 0 1
-1 0 -1
0.971354 0.209317 1
-0.971354 -0.209317 -1
0.906112 0.406602 1
-0.906112 -0.406602 -1
0.807485 0.584507 1
-0.807485 -0.584507 -1
0.679909 0.736572 1
-0.679909 -0.736572 -1
0.528858 0.857455 1
-0.528858 -0.857455 -1
0.360603 0.943128 1
-0.360603 -0.943128 -1
0.181957 0.991002 1
-0.181957 -0.991002 -1
-3.07692e-06 1 1
3.07692e-06 -1 -1
-0.178211 0.970568 1
0.178211 -0.970568 -1
-0.345891 0.90463 1
0.345891 -0.90463 -1
-0.496812 0.805483 1
0.496812 -0.805483 -1
-0.625522 0.67764 1
0.625522 -0.67764 -1
-0.727538 0.52663 1
0.727538 -0.52663 -1
-0.799514 0.35876 1
0.799514 -0.35876 -1
-0.839328 0.180858 1
0.839328 -0.180858 -1
-0.846154 -6.66667e-06 1
0.846154 6.66667e-06 -1
-0.820463 -0.176808 1
0.820463 0.176808 -1
-0.763975 -0.342827 1
0.763975 0.342827 -1
-0.679563 -0.491918 1
0.679563 0.491918 -1
-0.57112 -0.618723 1
0.57112 0.618723 -1
-0.443382 -0.71888 1
0.443382 0.71888 -1
-0.301723 -0.78915 1
0.301723 0.78915 -1
-0.151937 -0.82754 1
0.151937 0.82754 -1
9.23077e-06 -0.833333 1
-9.23077e-06 0.833333 -1
0.148202 -0.807103 1
-0.148202 0.807103 -1
0.287022 -0.750648 1
-0.287022 0.750648 -1
0.411343 -0.666902 1
-0.411343 0.666902 -1
0.516738 -0.559785 1
-0.516738 0.559785 -1
0.599623 -0.43403 1
-0.599623 0.43403 -1
0.65738 -0.294975 1
-0.65738 0.294975 -1
0.688438 -0.14834 1
-0.688438 0.14834 -1
0.692308 1.16667e-05 1
-0.692308 -1.16667e-05 -1
0.669572 0.144297 1
-0.669572 -0.144297 -1
0.621838 0.27905 1
-0.621838 -0.27905 -1
0.551642 0.399325 1
-0.551642 -0.399325 -1
0.462331 0.500875 1
-0.462331 -0.500875 -1
0.357906 0.580303 1
-0.357906 -0.580303 -1
0.242846 0.635172 1
-0.242846 -0.635172 -1
0.12192 0.664075 1
-0.12192 -0.664075 -1
-1.07692e-05 0.666667 1
1.07692e-05 -0.666667 -1
-0.118191 0.643638 1
0.118191 -0.643638 -1
-0.228149 0.596667 1
0.228149 -0.596667 -1
-0.325872 0.528323 1
0.325872 -0.528323 -1
-0.407954 0.441933 1
0.407954 -0.441933 -1
-0.471706 0.341433 1
0.471706 -0.341433 -1
-0.515245 0.231193 1
0.515245 -0.231193 -1
-0.537548 0.115822 1
0.537548 -0.115822 -1
-0.538462 -1.33333e-05 1
0.538462 1.33333e-05 -1
-0.518682 -0.111783 1
0.518682 0.111783 -1
-0.479702 -0.215272 1
0.479702 0.215272 -1
-0.423723 -0.306732 1
0.423723 0.306732 -1
-0.353545 -0.383025 1
0.353545 0.383025 -1
-0.272434 -0.441725 1
0.272434 0.441725 -1
-0.183971 -0.481192 1
0.183971 0.481192 -1
-0.0919062 -0.500612 1
0.0919062 0.500612 -1
1.23077e-05 -0.5 1
-1.23077e-05 0.5 -1
0.0881769 -0.480173 1
-0.0881769 0.480173 -1
0.169275 -0.442687 1
-0.169275 0.442687 -1
0.2404 -0.389745 1
-0.2404 0.389745 -1
0.299169 -0.324082 1
-0.299169 0.324082 -1
0.343788 -0.248838 1
-0.343788 0.248838 -1
0.373109 -0.167412 1
-0.373109 0.167412 -1
0.386658 -0.0833083 1
-0.386658 0.0833083 -1
0.384615 1.16667e-05 1
-0.384615 -1.16667e-05 -1
0.367792 0.0792667 1
-0.367792 -0.0792667 -1
0.337568 0.15149 1
-0.337568 -0.15149 -1
0.295805 0.214137 1
-0.295805 -0.214137 -1
0.24476 0.265173 1
-0.24476 -0.265173 -1
0.186962 0.303147 1
-0.186962 -0.303147 -1
0.125098 0.327212 1
-0.125098 -0.327212 -1
0.0618938 0.337147 1
-0.0618938 -0.337147 -1
-1.07692e-05 0.333333 1
1.07692e-05 -0.333333 -1
-0.0581615 0.31671 1
0.0581615 -0.31671 -1
-0.110398 0.288708 1
0.110398 -0.288708 -1
-0.154926 0.251167 1
0.154926 -0.251167 -1
-0.190382 0.206232 1
0.190382 -0.206232 -1
-0.215868 0.156247 1
0.215868 -0.156247 -1
-0.230974 0.103635 1
0.230974 -0.103635 -1
-0.235768 0.050795 1
0.235768 -0.050795 -1
-0.230769 -1e-05 1
0.230769 1e-05 -1
-0.216903 -0.0467483 1
0.216903 0.0467483 -1
-0.195432 -0.0877067 1
0.195432 0.0877067 -1
-0.167889 -0.121538 1
0.167889 0.121538 -1
-0.135977 -0.14732 1
0.135977 0.14732 -1
-0.101492 -0.164567 1
0.101492 0.164567 -1
-0.0662277 -0.17323 1
0.0662277 0.17323 -1
-0.0318831 -0.173682 1
0.0318831 0.173682 -1
6.15385e-06 -0.166667 1
-6.15385e-06 0.166667 -1
0.0281431 -0.153247 1
-0.0281431 0.153247 -1
0.05152 -0.13473 1
-0.05152 0.13473 -1
0.0694508 -0.112592 1
-0.0694508 0.112592 -1
0.0815923 -0.088385 1
-0.0815923 0.088385 -1
0.0879462 -0.063655 1
-0.0879462 0.063655 -1
0.0888369 -0.0398583 1
-0.0888369 0.0398583 -1
0.0848769 -0.018285 1
-0.0848769 0.018285 -1
0.0769231 3.33333e-06 1
-0.0769231 -3.33333e-06 -1