I am currently practicing the ropes of WEKA modelling with the free UCI breast cancer .arff
file and from the various posts here I was able to tweak it's accuracy ranging from 63% to 73%. I use WEKA 3.7.10
in a Windows 7 Starter machine.

I used attribute selection to decrease the number of variables using
InfoGainAttributeEval
withRanker
. I chose the topmost five with the following result:Evaluator: weka.attributeSelection.InfoGainAttributeEval Search: weka.attributeSelection.Ranker T 1.7976931348623157E308 N 1 Relation: breastcancer Instances: 286 Attributes: 10 age menopause tumorsize invnodes nodecaps degmalig breast breastquad irradiat Class Evaluation mode: 10fold crossvalidation === Attribute selection 10 fold crossvalidation (stratified), seed: 1 === average merit average rank attribute 0.078 + 0.011 1.3 + 0.64 6 degmalig 0.071 + 0.01 1.9 + 0.3 4 invnodes 0.061 + 0.008 3 + 0.77 3 tumorsize 0.051 + 0.007 3.8 + 0.4 5 nodecaps 0.026 + 0.006 5 + 0 9 irradiat 0.012 + 0.003 6.4 + 0.49 1 age 0.01 + 0.003 6.6 + 0.49 8 breastquad 0.003 + 0.001 8.5 + 0.5 7 breast 0.003 + 0.002 8.5 + 0.5 2 menopause

After removing the lowranked variables, I proceeded to create my model. I chose Multilayer Perceptron because it was a required algorithm from the journal I was basing my study on.
The suggestion of Bernhard Pfahringe to use 0.1
for the learning rate
and momentum
and the factors of exponential numbers 1, 2, 4, 8, for the hidden nodes
and epoch
and so on.
After a few tries with the method, I noticed a pattern of using 2 for the hidden layers and a decimal equivalent of a binary number ie. 512, 1024, 2048, … that results to increasing accuracy. For example, a hidden node
of 2 with epoch
1024 and so on.
I have a varied series of results but the highest one I got so far was with the following (using hidden node
2 and epoch
16384:
Scheme: weka.classifiers.functions.MultilayerPerceptron L 0.1 M 0.1 N 16384 V 0 S 0 E 20 H 2
Relation: breastcancerweka.filters.unsupervised.attribute.RemoveR12,78
Instances: 286
Attributes: 6
tumorsize
invnodes
nodecaps
degmalig
irradiat
Class
Test mode: 10fold crossvalidation
=== Classifier model (full training set) ===
Sigmoid Node 0
Inputs Weights
Threshold 2.4467109489840375
Node 2 2.960926490700117
Node 3 1.5276384018358489
Sigmoid Node 1
Inputs Weights
Threshold 2.446710948984037
Node 2 2.9609264907001167
Node 3 1.5276384018358493
Sigmoid Node 2
Inputs Weights
Threshold 0.8594931368555995
Attrib tumorsize=04 0.6809394102558067
Attrib tumorsize=59 0.7999278705976403
Attrib tumorsize=1014 0.5139914771540879
Attrib tumorsize=1519 2.3071396030112834
Attrib tumorsize=2024 6.316868254289899
Attrib tumorsize=2529 5.535754474315768
Attrib tumorsize=3034 12.31495416708197
Attrib tumorsize=3539 2.165860489861981
Attrib tumorsize=4044 10.740913335424047
Attrib tumorsize=4549 9.102261927484186
Attrib tumorsize=5054 17.072392893550735
Attrib tumorsize=5559 0.043056333044031
Attrib invnodes=02 9.578867366884618
Attrib invnodes=35 1.3248317047328586
Attrib invnodes=68 5.081199984305494
Attrib invnodes=911 8.604844224457239
Attrib invnodes=1214 2.2330604430275907
Attrib invnodes=1517 2.8692154868988355
Attrib invnodes=1820 0.04225234708199947
Attrib invnodes=2123 0.017664071511846485
Attrib invnodes=2426 0.9992481277256989
Attrib invnodes=2729 0.02737484354173595
Attrib invnodes=3032 0.04607516719307534
Attrib invnodes=3335 0.038969156415242706
Attrib invnodes=3639 0.03338452826774849
Attrib nodecaps 6.764954936579671
Attrib degmalig=1 5.037151186065571
Attrib degmalig=2 12.469858109768378
Attrib degmalig=3 8.382625277311769
Attrib irradiat 8.302010702287868
Sigmoid Node 3
Inputs Weights
Threshold 0.7428771456532647
Attrib tumorsize=04 3.5709673152321555
Attrib tumorsize=59 3.563713261511895
Attrib tumorsize=1014 7.86118954430952
Attrib tumorsize=1519 2.8762105204084167
Attrib tumorsize=2024 4.60168522637948
Attrib tumorsize=2529 5.849391383398816
Attrib tumorsize=3034 1.6805815971562046
Attrib tumorsize=3539 12.022394228003419
Attrib tumorsize=4044 11.922229608392747
Attrib tumorsize=4549 1.9939414047194557
Attrib tumorsize=5054 5.9801974214306215
Attrib tumorsize=5559 0.04909236196295539
Attrib invnodes=02 5.569516359775502
Attrib invnodes=35 7.871275549119543
Attrib invnodes=68 3.405277467966008
Attrib invnodes=911 0.3253699778307026
Attrib invnodes=1214 1.244234346055825
Attrib invnodes=1517 1.179311225120216
Attrib invnodes=1820 0.03495291263409073
Attrib invnodes=2123 0.0043299366591334695
Attrib invnodes=2426 0.6595250300030937
Attrib invnodes=2729 0.02503529326219822
Attrib invnodes=3032 0.041787638417097844
Attrib invnodes=3335 0.008416652090130837
Attrib invnodes=3639 0.014551878794926747
Attrib nodecaps 4.7997880904143955
Attrib degmalig=1 1.6752746955482163
Attrib degmalig=2 6.130488722916935
Attrib degmalig=3 6.989852429736567
Attrib irradiat 8.716254786514295
Class norecurrenceevents
Input
Node 0
Class recurrenceevents
Input
Node 1
Time taken to build model: 27.05 seconds
=== Stratified crossvalidation ===
=== Summary ===
Correctly Classified Instances 210 73.4266 %
Incorrectly Classified Instances 76 26.5734 %
Kappa statistic 0.2864
Mean absolute error 0.3312
Root mean squared error 0.4494
Relative absolute error 79.1456 %
Root relative squared error 98.3197 %
Coverage of cases (0.95 level) 98.951 %
Mean rel. region size (0.95 level) 97.7273 %
Total Number of Instances 286
=== Detailed Accuracy By Class ===
TP Rate FP Rate Precision Recall FMeasure MCC ROC Area PRC Area Class
0.891 0.635 0.768 0.891 0.825 0.300 0.633 0.748 norecurrenceevents
0.365 0.109 0.585 0.365 0.449 0.300 0.633 0.510 recurrenceevents
Weighted Avg. 0.734 0.479 0.714 0.734 0.713 0.300 0.633 0.677
=== Confusion Matrix ===
a b < classified as
179 22  a = norecurrenceevents
54 31  b = recurrenceevents
My question is how can I raise this data's accuracy to at least in the 90% mark?
Do I have to do filtering, use another pattern of MLP input parameters?
I plan to have another set of data that I will use after I've learned how to do this (it has around 50 variables and 100,000 instances).
Best Solution
There is obviously no good answer for such a question, but I will give you some more or less general hints for using MLP: