Improved Detection of Breast Cancer Nuclei Using Modular Neural Networks Analysis of Nuclei in Histopathological Sections with a System that Closely Simulates

he c v a l u a l i o n of immunocylocytochemistry allow the direct detection T chemically slaiimd histopathological of receptors in rnulinely processed sections prcscnts a ctiinplcx problcm due histological seclions of tumor lissiie L21. to many variations that arc inhercol i n thc In such preparations, however, the rcsults incthodology. In this rcspccl, many asare suhjcciivc and at hcsl caii only hc pccts [if iinmunocylochcinistry remain seini-quantitative; thus, immunocytounresolved, despite the l'acl lhat results cheiniciil metliuds are dis;idvantageous a s may carry imporkin1 diiigiiostic, progiioscompared to biochemical methods, which tic, and therapeutic information. In this arrender a quairtilative result in units of rcLick, a iiiodular neural iictwork-bascd ccplors per weight of tissue [3,4]. Dcspite approach to the dctcclion and classilicathis drawback, iminonocytocheinic~il tion of lhrcast cancer iniiclci stained for stcincthods havegaincd wide acceptance beroid receptors i n histopathological ciiiise thcy are lcss costly, easicr Lo pcrsections is described a n d cvaluated. Tlic form, need sinall amounts of tissue, and system, nainetl biopsy aiialysis suppor~ inosl iinportantly, can be carried out on system (DASS), was designcd s o that it routiiic histological seclioiis. In this resimulalcs closely the a incnt proccspcct, irninuiiocytochcinistry allows the siiiiiiltiiiicous iissessinent of tunor morphology and liormonal staining L o be pcrh m c d 011 serial sections. In addition, this lcchnique lhiis revealctl Ihe existence of practiced by histopalhologists.

Breast cancer is the major malignancy aF fccling Lhc female popukition in induslri-ali7.edcountries; it is estimaled thal one-third of these patients cvcnlually die bccauscoflhis disease.There is, however, a witlc vari;itiun i n the length or survival of individual paticnls, wilh some surviving several ycars following diagiiosis.11 has hcen widely rccogniecd that palient s u r v i v d is ossociatcd with clinicopathulogical assessniciil cif several hacbetter known as prognostic factors, h enable clinicians to predicl individual palient survival and illso choose appropriate inudes of treatmcnl.
Of proini ncnl i in portancc among prognuslic factors is the hormonal status of tilmor cells, which involves analysis for estrogen and progcsteronc receptors.Traditionally, lhese analyses were pcrSormcd using biochcmical melhods [ I , 21, hut more recently rapid advances in innnuno-hclcrogeiicous shining within tiiinor niiclci-a Finding that was not previously appirciil hy biochemical analysis 12,51.
In orilcr to improve the predictive accuracy or imnitinohistocheinical data wilh regard to cslrogcn and progesterone receptors, several invcstigalors have dcviscd diagnostic schemes, such as Ihe Hi-score [21 or the diagnostic index 161.Thesc schemes are bascd on the combined evaluation 01 two variahlcs; namcly, the slaining intensity of individual tumor nuclei and tlrc percentage of cells that are stained iit cach ititcnsily class.Thc aim of these manual diagnostic schemes is to enable a semi-quantitative assessinen1 ofthc microscopical imagcs, [lie interprctalion of which is only subjective when carricd out routinely.Moreover, there is currently a major effort in sk"lization and quality assurance i n histopathology [71.In this context, commercial image-analysis sys-terns, such as SAMBA and CAS, have been developed and are hcing applied for thc.quanlitatioii of imintinocylocliciiiicnl images [8-121.Thcsc systems, although dedicated, do not aclually simulate Ihc inaiiual procedure employed by liiiiiiaii experts, which involves the c1;issification of individual tumor nuclei.CAS and SAMBA use aglobal approach to classify shined nuclei hy introducing thrcshold levels that distinguish hetwccii specific staining arid background (nonspecilic staining).In this respect, the devclopiiieiil and application of coinputer-aided systems such as BASS 113,141, which reproduce and enhance the experla' ability Lo detect ohjccts of intcresr (stained nuclei in this case) on an individual ralhcr than H global basis, iiiay enable a more quiintitalive iissessiiiciit of iiiiiniinoliistoclicinical results, aiid therefore improve their predictive accuracy.

Materiul
Cryoslal sections from frozen biopsies of breast c ~i ~i c c r pnticiils, 4 I in total, were cut at 6 pin aiid placcd un poly-L-lysine coaled slides.lhe sections were lixetl and iininunolahellcd using specific antibodies lo estrogen and progesterone receptors (ER-ICA/PgR-ICA k i t s , Abbott, Gcrmany).Positive iiucleiir staining (brown color) was visualized using tlic strcpt ARC kit linked to peroxidase (DAKO, Denmark).Subseqiientiy.sections wcrc countersteined with lheiniitoxyliii to higliliglil unlabeled iiuclei which stained blue.

Modular Neural Network System
T o assign a diagiiostic index to a hopsy specimen, BASS iiiiplciiicnls a iiioclu l a r approach t l i a l resembles t h e algorithm used by liuliian experts.BASS  Gwssians is the facl that tlie filters arc localked both in the spatial and frequency domains.Morcover, the fillers can hc conveniently sciilcd to acam"daie opiimal response in a dcsired range of spatial scales.Here, the receplive Siclds are applied as general templates lo enhance matching local imagc slructurc and sup- press the resl.Due lo the lypical center-surround slructurc o l Gaussian rcceptivc fields, the matching proccss also depends on the iminediale ncighborhood of nuclei that arc supposed to hc inalchcd hy ilie Tiller.This leaturc givcs control over how well il nucleus Inas to be separaled frvm other iinagc slructures in order lo bc detccled.A dcleclor always incctls to decide when an evcnt is "lo he detected" or "not to he delectcd."A squashing function, initializcd using image statistics as part or the iterative process oT the RFS algorithm, acts as asoftthreshold.Tlie runclion clividcs the image pixels into background pixcls and pixels belonging to nuclei by gl-adually transforming both sets closer lo the extreme values.
The 1WS niiclci dctcciion ;tlgorithm autuinalically adapls I I I local and globel iinaging conditions.The user may interxi with lhe ;ilgorithm via twu par;imeters, which indicate thc approximale nuclear s i x in the biopsy irnege and tlic distance to neighboring image slructures such its other nuclei.However, tliesc parameless wcre fixcd Tor a11 images at the same value to avoid uscr intcractivn in the cxperiments.In the following subsections, the rnajor stcps of the RPS detection alporithm are prcsented, wliilc a more clctailed accounl can be found in I 131.
Step 1: Cmiwt-t Color i m q e to Optical

Density I m q e
The original RGB color iinagc is transformed into a scalar array I using the Y channcl(optica1 density) olthcRCB-YIQ transform 1211

Field Filtpr
The reccptivc lield array is given by lhe Ibllowing equation Kf(*,y) = I1 (2) where B, a,&, p2, and .lareconslants.The parametcrs p, and p2 determine the scnsilivily of the liltcr regarding the range of object sizes and wcrcaetto2.5 andl.5,respcclively:/3, a, and .I are automatically determined as given in 1131.Lrrs scale, ufl:~et, and incl arc ;idjusicd according to the current image array antl thc prcvious paramcter values.Thcn, the image is convolved wiih the rcccptivc field and then squashcd.Within thrce i t e m lions, ihc iinagc iniensity distribuiion takes on a bimodal shape, cleerly indiciiting the prcscnce of two pixel classes: (i) backgroimd, and (ii) candidate nuclei.Rcpcated application of the receptive field and the squashing fuiiction to the image ensures that object detection is mostly dependent on object geometry, antl not 011 object intensity.Thc squashing function is defined as: scale where sculc determines ihe range of the function, incl determines the inclination, and qfjler dctcrmincs the offset along ilic abscissa.Thcsc parameters arc again determined automatically, as described in (131. Step 4: Tlzreshold Ilimodal Histognrm Thehistograiiivcctoruf I, (Le,, theimage array aFicr the lhird iteration) is smooihcd wiih a moving average filter.The threshold value T i s set equal to thc hisiogram bin with ihe minimum amount of pixels between lhe two modes of the histogram: T = niin ,,,,,,, .(hWU6 1 .a,,,..., a,,,,.,,,~.,]) ( 5 ) where hist(1) returns the histogram of I, a, l,...,a,,,, ,,,,.~, = 10.3 scale^' dcfine thc moving averagc filtcr coefficients, and min ,returns thc minimum between two maxima, which is uscd toobtain the corresponding lhreshold intensity T. T i s used to segmcnt the sample image into background antl candidate nuclci.The candidate nuclci in t h e i m a g e (i.e., all connected sets of pixels) arc recorded in an object list.
Step 5: Revise the L k t qf Detecrcd Nuclei The nuclei ccnter locations are computed by determining the ccnter of gravity of each candidate nucleus and lhen returned.
Detection of Nuclei: The Feedforward Neural Network

(FNN) Module
The algorithm presented in this section deiects the localions of nuclci in hiopsy images, based uii a supervised neural nctwork.Block-based proccssing of the images is adopted, lbllowcd by an SVD OF each block.The most important singular valucs are fed as inputs to a nciwil networkclassifier, which, in him, dotermines the likc.lihoodtliai the original image block contains a nuclcus.The neural network is traincd in a superviscd oiodc, thus allowing lhe knowledge pl-uvide<l by the cxperls to be included in its design.
In general, features arc exlrticted by applying a kansformation to tlie image blocks, so that the image space is mapped onto another space, which is assumed to he morc suitable For classification.Commonly uscd fealurcs include the inem and variance of the values of pixels in each block, momcnts, and other corrclation-type or higher order statistical parameicrs, coefficienls of the Fourier or the discrete cosine transform (DCT) of the block, as well as delerministic paramctcrs related lo the size, color, and conncctivity ofihe block 122,231.Generally, the above mapping is not intended to be one to onc: i.e., there is soinc loss of information when moving from the original image to the featurc space, mainly to simplify the classificaiiun iask.In particular, those feaiurcs that do Given the diagonal matrix and using a column vector e = (!,I ,...,I)" of sizc 11, wc can generate the singular valuc vector (svv) of matrix A by postinidliplying inatrix A wiih vcctore: Underthcconstrainlh, > h , >...> h,, matrix A and vector svv are unique Cor a given nialrix A 1251.
Apart from the above, it should be mentioned that singular values arc inscnsitive lo small changes in matrix A. Assuming that matrix A denotes an image block, svv and consequently the neural network detector are insensitive to small changes of pixel values caused by noise or different illuniiiialion conditions.More-over, singular valucs remain the same evcn if the image block is rotated, transbaled, or transposed.The above properties are highly desirable in the detection task, wlicre the piisition, and not lhe orientation, of ihe nuclei is required.The proposed scheinc is composed of the following steps.
Step I : Color.lniugi, IO Optical Density

Irriag~ Conversiorr
This stcp is identical to that of the KFS module, exccpt that the optical deiisity image is not inverlcd [see Eq. (I)].

IElE ENGINEERING IN MEDICINE AND BIOLOGY
iinagc Y .The Lhresholded image Z is given by: Sfep 3: S V Eq;l,aiisiuii and Feetforwnrd

Neural Neiivurk Clossificutiniz of l m q e Hhcks
This stcp is bascd on a block-oriented architccture, as illustraied in Fig. 2. In particular, the preprocessed images are raster scanned both hori~ontelly and vcrlically using a scanning step 01 k pixels and a blocksiaeofNxNpixels.Thus, thciinage is seperaled inlo overlapping blocks,R(i, ,/I, tlic nuclei posiliuns.Howevcr, this is not Lhc case, due lo Ihc dilfcrenl nuclear sizes and to overlapping nuclei.Iiistead of isolaled points, the oulpiit image gencrally conlains clusters of points (i.e., blocks) within, iind possibly around, tlie arca of llic nuclei.In order lo achieve the final classilication cif thc blocks, selecLion of thevnlucoEaglobal llircsliold~isneedcd.
On tlic other hand, the exact positions of tlie iiuclei centers tnusl be calculatcd.
First, lhc global threshold T, which is actually a paramelcr indicating how conscrvalivc the dcteclor is, can bc selected by [lie user, dlowing exlernal control of the algorilhin perl'iirinance.Thcn, tlie exact positions of nuclei are computed as the local inaximii of tlic output iinagc pixel valucs, according to the rollowing rule: "Lctthepositionofthe 'positive' pixel witli t h e b i g g e s t v ~l u c , williin ii prcspecilicd neighhorliood of each 'positivc' pixel of the oulpot image, be selccted as the center of the nucleus".
where Ti2 > T , , ~, , T,, = T , and T,> < "maximum valoe of 0." 2. Each binary inask S,,(iJ is processed using ii iniorphological shrink opera t o r 1261 I F i g .2(c)] to g e n e r a t e corresponding isolated points, while pre- serving the numberofcluslers of points in the mask; i.e., the tnorphological shrink operator preserves tlic Eider number 1271.
1.The resulting nutpols, say C,,, arc [Fig.The selection of the various parameters uscd in the FNN dctectioii module is prcsentcd in Appendix B.

Combination of Detection Modules
Since the RFS and FNN modulcs work on different principles, our aim was to lest their performances, both individually and combined.In this study, the dclectioii results of thc RFS and FNN modules were combined, based on modified logical OR and AND operators.ORDt ( R I 3 .OR.FNN) and ANDDt (RFS AND.FNN) modules werc evalualed along side the individual RFS and FNN modules.The scinantics OS the logical opcralors had to he expanded to account for the rather limited spatial accuracy of the detection events generated by each module.Therefore, a fixed tolerance WAS uscd to addrcss the iradc of[ hclwecn using the precise nuclei locations dclectcd by the two modulcs, and using the less strict notion of proxirnitv of detected location to decide whether 0% Of nuclei, score 0. are negative (very light gray, Original image: blue), score 0 50% of nuclel, 8wie 2, are weakly stained (light ray original image: blueibrown), score 1: L?l('/l 3: COlll/JltfP I)kjin(J.Sti?Id&Y V i e diqiioslic index is compukd according to the iniiniiiil seini-quantitative The iissessniciil rcsulls of tlic preccdiiig slcps iirc finally slorcd in Ihc diilabase ol'c~iscs using the retrieval interfacc 1151.

System Validation
U l h a l c l y , eny systcin or cxpcrt can he said to pcrfnrin successfully il llic niiclci propiirtions iii iiii iiiiiige arc cstiinalcd ;iccuralcly and rcliiihly, since llic diagnoslie iiidcx i s b;ised on this cstiiiialc.However, at Iprcscnt, il i s dilficult I O evaliiiite eilhcr tlie pcrfori1i;incc of lhc cxpcrls or llic systcnis, due lo Lhc unaveilabilily of iiiiivcrsiil gold skindads ill Ihc nuclei dc-Icclin~i and niiclei classilicatioii (i.e., diiigiiostic iiidcx) IevcIs.Pirslly, the system was evaluated iil the iiiiclci detcclion level by coiiipxing its perl'orlnaiice to that or two liiiiiiaii experts, iisiiig tis the basis 200 io 300 nuclei per case, which wcrc marked by the experts.Sccoiidly, llic syslcni was cvaliiatctl at llic nuclei classilicalion level by coinparing thoseclassificcl by BASS to lcntioii was given to calibration aial data acquisition stand;irds.
itc ii basis Sor comparison at the iiiiclci Icvcl, a siiiiill circular probcolninc pixels rliainctcr was plnced centrally on top ol'cach IiucIciis delccteil by the syslem iiiodulcs.Indcpcndently.two cxpcrls placcd ilie siiiiic-sixd ~~O I J C S manually where they perceived tlie nuclei centers were located using the inoiise.I n addilioo, the prohes were color-codcd, depending on the shining inleiisity class coinpiited by the BASS classificalion algorithm dcscribed earlier, or assigiicd hy llic expert.
As a result cif this proccilure, lor each module iiiid each expert (i,e., the comhinalions) one mask image w diagnostic index for all mask images was euloniaticelly computed according to Tab l e 5, u s i n g rhc above-described color-coded inask images.
A propurlions i i f all liopsy images.This cocfficient makes no assuinptioiis ahout llie underlying probahility distrihutions, and since proportions arc analyzed, llie measureincnls arc iiidcpendent of thc iihsolutc nunibcrs of detectcd inuclci.However, hias towerds one or the olhcr nuclei class or staining intensity, i s rcgistcred.This i s a relative bias, due to the lack of gold skindards.The Spearman's rank-order c o r r c l t h i i coefficient is given by: where x and yare vectors containing Ihe proporlions o l a particular staining intensity ~IIISS for iill lnicipsy images, and /<x, and /<jj arc riiagnitudc~hascd ranks among x and y. respectively.
This part of lhc study was hascd o n 29 iinagcs that were selected by the experts on the hasis of clarity of color perception and minimal overlap displayed between

IBE ENGINEERING IN MEDICINE AND BIOLOGY
IonuoryIFebruoiy 2000 ' Table 3. Spearman's rank order correlatlon analysis.Comparison of staining intensity class proportions for all 57 images (all entries are slgniflcant a t t h e 1% level).

Detection Example
Figuse 5 shows ii subrccginn OS a breast caiiccr biopsy iinagc inaikcd by all experts and modules, and their combinetions.The inotlules generally delcct inose nuclci in thc biopsy irnngcs than 110 both experts.The siiiiill rectangle illustratcs the pcrCormance ofthe OR and AND operators.lo particular, the individual exp r t s and the inodulcs liave iiiaskcd the niiclcus towards the left side ofthe rcctanglc, buteachexpertassigiicdadilccrent nuclear slaining intensity class (differencc is only visunliscd in color).In addition, Ex1 de-0.9220tected two innre nuclei thal Ex2 did not mark.The result is that llie OR operator includes onc of the locations and llic cosrespoiiiliiig iiticlcar class lor the nucleus dctccted by both experts, and the two nuclei not detected by Ex l .In conlrast, the AND operator ignores all the iiuclei in thc sinaller 1-cctanglc, since there is disagreeincnt regarding nuclci locations as well as their staining intensity classcs.

ROC Analysis
Table I tabulates llie delcction pcsfosmince regarding the 57 images in the dat a b a s e .In this t a b l e , t h e second component is takcii t o bc tiic gold slandwd.I1 is iiotctl tlial if component I were lo he clioseii as the gold standard, the nuinbess would only havt: to he iiitcschanged, due to thc definitions of SS and PPV.The table consists of fivc sections, ordcrcd hy the applied laboratory gold standard.However, the PPV values arc at l c a s l 12.7% higher than the bcst pcrforiimnces by any single module or ORDt, reaching 82.4% which approaches l'avorably the pcrSormancc OS the experts.
The modules and lheir cornhinatiun perform poorly regarding the PPV vdliie when compared lo ANDEx (section V).ANDDl perlbrins best regarding the PPV, while ORDt pcrform hcsl regarding the SS.Since the number of nuclei inarked and classified ill agreeincnt is lowcr than the numbers irom iiidividual experts, the PPV values for both the inodulcs aiid the individual cxperts arc very low.On avcrage, the experts agree only in about 53% ufthc cases (sectioii I, llxl -ANDEx, Ex2 -ANDEx, (52.8 + 54.7112) on the localion, while ANDDl agrees only with a inaxiinum of 40.7%.It is iiotcd that [lie delinition of the AND opcratur requires detected nuclei not uiily to coincide spatially, but also lo be labeled identically.

Confiisiori Matrix Comparison Regarding the Assignment of Diagnostic Indices
Thcresullsshown inTable4reflcctrlie pcrformancc of experts and modules with respect tn a subset of 29 images, tis cxplained earlier.The row entrics show the percentage of images with a particular diagnoslic index assigned inaiiually (iis given in Appendix A) to the biopsy slide versus the ones coiiiputcd by BASS.The last row of the lablc displays the avcmge pcrcenlagc of correctly graded images per diagnostic index.The miiiii characteristic of the confiisioii tnatriccs considered is the averagc percentage o f correclly graded iinages per diagnostic index class (CCI).As inentinned above, the actual distrihulion of diagnostic indices was 17%.176, 1790, 3S%, and 14% starting with the "0' and ending with the "4+" diagnostic indcx entrics, respectively.The reprcscntatirm of the contiision matrices

Discussion
Histopathological sections of breast c~inccr nuclei iminninocylochcinicailly stained for sternid receptors arc roulincly reported by experts, based on the microscopical evaluation of iiuinhers o f iiuclci stained at particular inkmities of brown color.This study sliows that detection and clwsiCicatinn o1 individu;il nuclei in histopalhological sections can be reliably perfnrmcd by lhc BASS modular neural network system in an iiccumtc and consistent miinner.BASS also facililatcs interaction with experts aiid (U this eflect, the

IEEE ENGINEERING IN MEDICINE AND BIOLOGY
second-run validation rcsults indicate h i i t this interaction is constriiclivc, since it was demonstralcd that lhc modules correctly detect considcrahlc numbers oiiioclei that were not initially dctccted by the expcrts.Morcover, sincelhc system simulates detection and grading strategies of hurnaii experts, it will enable tlic formulalion of inure efficient slandardimtion criteria i n t h e a s s e s s m e n t o f iminunocytochcinicelly slaincd histopiithologiciil sectiniis 141.
The ANDcd R F S -F N N inotlulc, ANDDt, I c d s to the hest overall results in ternis of detection accuracy for Lhc cliaginostic indices.I1 achieved the highest PPV, a s cooiparcd to OREx, aller the second-ru~i vinlidalion (83.6%), and the highest averagc accuracy l o r correctly assigning diagnostic indices lo the images (69%).However, llic SS is lower Ihan for any other combination of iniodulcs (6 I.I %').It should he noted that although Ihc KI'S module inatchcs the overall per-Comiaiice of ANDDt for the diagnostic indices, its valucs for SS and 1'1'V wcrc 78.1% and 75.0%, respectively.
l h e prcscnt data show that a high PPV value is criliCal for obtaining a good performance with respect to the diagnostic an be seen when compnring llic cxperls and BASS combined detection and classification modules.On the other h;ind,otirdntashow that theSS valuedoes appear to bc 21 less important factor and lint dircctly related to BASS pcrfiirinance in computing diagnostic indices.The experts showed a lcndencp to overscure, as is dciiionslratcd by tlic diagnostic index confusioii inalrices, while the cumhinalions of detection ancl classification niodules both overscore and underscore.l'his tendency of the experts to overscore may bc explained by tlie observation thal llie Spearmiin rank correlatinii valucs were higher t'nr moderate to very strong nuclei.However, the Spcarinan rank correlalion values for the BASS system (RFS, PNN, ORDt, ANDDt,combined with theckissificalion module) lie above 0.87 (except for the 0.76 cosrclation value fos the wcak nuclei regarding IIFS-FNN), which iim plies that the modules and/or their coinbinations pcrrosm c o n s i s t e n t l y ancl uniformly.111 addition to higlrcr accuracy and grcaies ohjectivity, iiilagc-analysis sysleins should also po lhan that required hy huinan expcsls to perfurm similar tasks.BASS i s able lo pesfiirm the analysis 111 onc imagc, on inemge, in less than nne minute (200 MHz Intel Pentiiini PC, 32 Mhylc RAM).This time-span compares favorably with the limc needcd hy liumiin expcrlr to perform similar tasks. In based on a singtilar valuc decomposilion (SVD) 01 image b l o c k s wiis c o m b i n e d w i t h the prc-existing receptive ficld/squnsliing runclion inutltile (RFS).[n addition, the database was cxpandcd and a human expert was used lo validalc BASS diagnostic pcrformance.This resulted in a marked improvement i n the predictive valiic Database t procccds lo grade a biopsy iinagc by l'irsl Ciiiding the localion or nuclei in llic iinagc (see Fig. I , Module I, Nuclei Delectioo: and tlicn classifying them into me of rive classcs according to staining intensity (sec Fig.I , Modulc II, Nuclei Classilication and Biopsy Scoring).Once thc nuclei are graded, a five-class nuclei proportion vector is computed, which is uscd to determine the diagnostic index according to thc mimual grading scheme (Appendix A).BASS also contains an image database relricval inlerkce (sec Fig. l , Module 111, Relricval IntcrTacej, which cm bc used Tor cvntenl-basccl biopsy iinagc retrieval fi-om a daiabase of cases 115 1.

Step 2 :
Compte the Receptive

If
X is an optical density image whose histogram values are limiled in Ihc inlerval [ a , h1, where n t 0, h 2 c, then liislogrxm strclching generates an image Y whose histogram lies in the extciided inierval 10, cl.Imagc Y is derived by the following 1l.ansliorination:Aftcr histogram strclching, the noisy hackground is smoolhed using an appropriate threshold, T , , which is a lunction OS 16 24 32 40 Size of Hidden Layer (Nodes) Size of Square Black (Pixels) 3. Percentage of correct hlock classification of training and test samples as a Siinction of ihe hlock sizc used to scan the imagc.

52 4 .
Perccntagc of correct hlock classification of training and test snmplcs as a fuiictiun of the size of the hidden layer of the neural network.

5 .
of N x N pixels [sce Fig. Z(a)l.These blocks are subscqiieiitly ~ransformed using SVD, producing singular valuc (SV) feature vrclors [sec Fig. 2(b)l.The SV feature vectors are then subjected to climensionalily reduclion 10 decrease furtherthe coinplexily of the algorithm and L o drop those singular values, which do no1 signilic;mlly contribute to tlie separabilily of the dataset.The truncated feaiure vectors arc fed into the !neural network [sce Fig. 2(b)l.Consequenlly, classification is pcrformed in the singular-value domain.Thc analog iiciiral network output Yalues for all blocks form an output intensity image 0 [lie siLe of which is (k x k) limes less than [hat of the original image; where 1; is the scanning siep, as shown in Fig. 2(a).Pixel valuer io the oulput image lhal arc close to unity show lhat the corresponding blocks in the inpul image have a high likelihood to bclong 10 a nuclcus.Step 4: Calculation of the Exact Ideally, lhe outpol of the neural detcclor would include isolated points indicating Exaniple for nuclei detection: (a) Exl, (b) Ex2, (c) OIIRx, (d) ANDlh, (e) RBS red inlo pr.inriiy k J ~i C O i OR jilirnctiort I , O I * = l , u D ( 1 1) niotlule, (0 BNN Module, (8) ORDt, (11) ANUl)t.The large square highlights the differeiiccs between experts and modules.The m a l l rectangle points to an example oS how the OK and the AND operator aSfect the generation cif comliincd results.D = u [ k W s12,Vj eI,,d(;, j ) > I ] , and d denotes the Euclidean distance and r is a parameter related to the approximate nuclear size.The output of the multiple input priority logical OR function is computed as follows: c = (...((C, @C2)B,c,) ... ec,).(12) thmc c1:issificd by (lie experts.I'inally, [lie diagnostic index, calculiiteil on the basis US the BASS iiuclei c1:issificalion rcstills, was coinparcd to tlic diagnostic iridcx oS each case ohtaiiied roulinely hy litiiiian experts.111 order LO iiinxiiiiizc the overall sysleiii pcrform:uice, particular al. -------- Figures 5(a) and 5(bj indicale the nuclei marked hy Ex1 and Ex2, whescas Figs.5(c) and 5(d) prcsent tlic detection ircsulls from OKBx and ANDEx.Figures 5(e-hj show thc detection performance lor the RFS module, the FNN module, thc ORDt modulo, and thc ANDDl module.The l a g c squasc inserted in ciicli Figure highlights a region of the imagc thal displays the perfosiiiancc of the experts and the inodules.
In this study, thc BASS delcction module has becn expanded lo include an FNN opcrating on a block-based SVD uTthc hiopsy images, besidcs thepre-existing R I 3 module [ 131.Both approaches utilize localizcd opcriiting principles that fcature rotalioiial invariance and insensitivity to noise.Due tu the SVD transformation or the image blocks, these propcrtics strongly rescmblc image encrgy distribulion fcaturcs tliat are rclated Lo tcxturc and edges.TlieRFS modulc with its rcceptive Cield Cilter is dcsigned to cnbancc or suppress rclativc diffcrenccs of iiuclci and background, and Ihus it is more relaled to avcrage properties of local image struclure.Since thc upgraded BASS systcm is based on two dirfercnl detection modules, inethodologies for combining lhc results frnm both modules need to bc invcstigated.The individual modules were evaluated as well as the logical conihinaiions OR and AND.In thc following sections,

A ' ~, or equivalently A"A, aiid h , are the singular values o I ma- trix A. The eigenvectors n, of AA", re- lated
with lhc eigcnvalues h:, are the coliiinns of matrix U, and lhc eigcnwztors vi of A''A are the cohiinns of matrix V.
: 1'2 = 2 20% of nuclei, score 1, are stmongly Stained (dark gray original image: dark brown).score 3: 1'3 = 3 5% of nuclei.score 1, are very slrangly Stained (very dark gray original image: dark brwn).score 4 1' 4 = 4 rotal Score: 11 Diagnostic Index: 3t the same nucleus was detected.Detected nuclei locations were considered to coincide if thc centers were located within a distance of nine pixels from cach other, as 6. 1,ight micrograph showing immunohistoche~nlcal staining of breast cancer nuclei for estrogen rcceptors (localized gray colnr; original image: brown color).This case was assigned the diagnostic index of 3+ based 011 Table 5.The compilation of the diagnostic index (3t) is shown below the micrograph (magnification x400).images all nuclei classes contribute equally to an marked by an expert.That particular expert had the hest average percentage oS The RBF nelwork structure consisted correctlygradedimagesperdiagnostic inof a single RBF unit layer that was fully 54 IEEE ENGINEERING IN MEDI(INEAND BIOLOGY where p is the input I'eature vector, w is the weight vcclor, disl is the Euclidean dis-iiic::wre, SI' is the spread coiisliiiit, :ind RI$/,' i s a Ciiussinn functioli.'I'he lr an5 ,I e! .lunction, ' TF [Eq.(1311, takes on i t s iniixiiiial v:iliic of unity wlicn its argo-StqJ 2: Ch.milj' Ecicil Nuc1cu.sA nucleus is classified inlo m e ollivc staining inlensity cIisscs (negative, weak, mod- detection event was deliiicd as tlic set of pixels bclonging to one u l the probes in llie image.If lwo prohcn Irom different iniask images overlapped, llicn h e two corresponding detection events were said lo cuincidc, and the corresponding iiiicleus was inlcrprcted 11, havc been detected in both inask images.In thc CLISO of oiic probc touching scveral uther probes, only one coinciding detcction cvciil was counted.Two hybrids, called OREx (Ex1 .OR.Ex?) and ANDEx (Ex1 .AND.ExZ), were derivcd with ii modified logical OR and AND uperalion Srom tlic individual experts' masks, 21s implemented in the OKDL aiid ANDDL modules.Systcin v;ilidation i y a s perhriiied using f i ~r mclhods.In parlicular, the dcteclion pcrlormance and the ,joint detection and classification performance of the BASS system wcrc assessed with the fol-(lruc positive) arc thosc nuclei marked in both the gold standard zind the image, and FM (false negative) arc lhosc nuclei that are marked i n the gold standard, but not in tlic image.Positive prediclive value (PPV) is the likelihood that tlie detection of a nuclcus is actually TP + FP) ( 16) wherc FI' (false positivc) are those nuclei which arc intirkcd in the image, hut in01 in the gold slao&ird.2.Spearinan's rank-order corrclalion coefficient 1301 was delermined to assess tlie joint performance of BASS tlctectioii and cliissiliciition modules coinpared to that ofEx I and Bx2, hmcd on thc staining lowing mclhods: I. Receiver-operator characteristic measures (ROC) were uscd to analyzc illdividual nuclei detection pcrfornlance of BASS dzlectiun niodules compared to that oftwoexperts.ROC ineasurcs arc useful to compare the detcclion perforinancc with respccl to individual nuclei.becausenoassumptionabouttlieunderlying prohebilily distribution of the detection cvenls i s made.Based on tlic definition ullhc dctection events, two measures, sensitivity (SS) and posilivc predictive value (PPV), were chosen to characterix the dctcclion performancc.Sensitivity is the likelihood that a nucleus will bc dctectcd if it i s also marked as a nuclcus in the gold slandard.It is defined as follows:SS = TP/( TP + P N ) (15)where TI' associaled willi a nuclcus inarked in lhe gold sLandard.PPV = T/'/( The McNcinar test 13 I ] was applied lu the diagnostic index resulls to find out if ificrs perfosm significantly dilfesent.The McNeinas tesl is based 011 the chi-square tcst.To perfosm the tcst, lwo counts musl be perlormed: (i) the number of those images that were correctly classified only by the one classifies (cormt,,,j, and (ii) thc numbcr o l images that werc correctly classified by the other classifier (count,, j. 'rhcn, the following comparison is pesformcd:

Table i ,
seclion 1, shows lliat Ex I has anSS of79.2% and a PPVol76.5'%, with Ex2 as the gold skindard.When Ex I and Ex2 iisc coinpared to OREX, their sensitivity values increases l o 83.8% and 8 I .I % , rcspectiveiy.For ANDEx, the PPV valucs of Ex I and Ex2 are just 52.8 and 54.74, with a high standard deviation of over 20%.The SS and the PPV values for lhc individual experts, whenTable I, sections TI lo V, show the dclection results of the individual and c o nhincd modules.The detection results for the individual modules are prcsentrd first, followed by ORDt and ANDDt.Thc SS andPPVvaluesoftheRFS andFNN modules (sections 11 and Ill), when comparcd to either Ex1 or Bx2, were in general quite similar, varying from 76.7 to 81.4% and 54.9 to 58..5'%, rcspectivcly.I t i s noted thal the PPV valiic~ oflhe individual RI'S and FNN inodulcs (scctions I1 and 111) rcmain about 20% hclow the 78% (section I), Ex1 -Ex2, (79.2+76.5)/2)avcragc level o i thc human experts.Thc SS values of the individual inodulcs verstis OREx (section IVj drnp only slightly.Thc PPV value.however, increases by 8.6% and 10.8% for the RFS inodulc, and 9.6% and 8.9% lor the FNN module.Combining the RFS and FNN inodules via the logical OR operation achieves SS valucs of over 91.6%, but tbc PPV valucs decrease by a inaxiinum of 8% bclow the lowest value of any individual module ior all combinelions wilh the cxperts (ORDl entries in sections II to V). ANDEx scores over 30% lowcr SS values than OREx.

17..t....n <ll,.*lr
an altempl to improve nbjectivity and offer mpid analysis, simie cominer-cia1 systems, such as CAS and SAMBA, rely on glohal discriminalion or structures of interest, between nuclei in this case, and background.These systems nieasuse percent slaincd susface area using global thrcsholding techniques.However, there i s disagreement among experts :ibout lhe uptimill scleclion of global lhseshrrlds, with thc choice being fixcd, manu;~l, os automatically set thseshold [321.BASS avoids the need for globel thresholdiiig and area measurement, since it detects, BASS was designed to simulaleclosely the tlclection and grading slrategics as fos exmiplc, included a variccy of dicignostic clues and detailed prior knowledge in a Bayesian belief netwosk to g ~d c prostale lesions, while Mangasarian, cl al. 1351, showed that linear progl-amming mcthods inay successfilly he applicdfur breast cancer diagnosis and prngnosis, based on coinpulcr-aided iinagc analysis iind other clinical data.It i s difficult to assess the Irue system pcrrormancc, with comparisons to other systems, in tlic absence 01 reliable and11 gold slandards 141.All cxperimeiits pcsfurmed hese had to be biiscd on labolatory gold standasds (i.e., either on lhc nuclei masking results 1som [lie ex-Expcrt 2 was clioseii as lhc source of supervisory infosination a1 the beginning o l this sludy.Howcver, the present data demonstsale tliiit both niodules RFS and FNN pcrfnrm consistently and accurately despite the facl lliat they wcrc using differetit merhodologics.In acldition, the coiil'usioii matrices arc Curthes proof that tllc BASS syslcm can iicliievc at leas1 similar results coinpared to the liuinan cxperls.Furthermure.BASS a second algnrilhm perfurming the identical task, hut cmploying a different appu0acli.I1 was shown that, indeed, tlic comhinntion ul~lie dciectioo ~~ii~dules RFS :mil ANN i n d m " I hcilcs than lhc indi- suse of ohjectivity and consistency of iiidividual cxpes~s, the Ccrinan Nzilional Research Ccntcs for Inl'osinalion Technology (GMD), St. Auguslin, Gcrinany as a rcseasch studenl in ihr area nf nonmonolonic logics and lsuth inainlenancc systcms.Fsom 1991 to 1992, he worked as a research assislaill ;uid rescarch associate in the Pclroleum Engineering Dcpastmcnl at Texas A&M Univcraily c m lhc detection o f fractures in sonic boreliule inragc data utilizing neural network technology.In 1993 he juined the Department or Coinpuler Science at the University of Cyprus as a research associnracticcrl hv liistomilliolosisls.Thus.cx-I pests may bc used lo supervisc ancl cvaluate the system at h e nuclei detection, the nuclei classificalion.and tlic diagnostic iiidcx levcls.In an effort to improve dclcctioii pcrfornmnce, BASS inuclci detection inodulc was expanded SOS this sludy lo include

1 , U L " L O * 1 < 1 1 1 .
Ncwcasllc upon Tyne, UK, in 1988, and llie PhD. in cIcc~runic engineering froin tlic Univcrsily of London, UK, i n 1992.During 1985-1092, hc was also working with the Cyprus Institute olNeurulogy imtl Genetics where he i s now a Visiting Sciiior Scicnlisl.He is currelilly iili ;issistlinl IJlOlcSSOl.wilh the Dcparlment of Compotcr Science :it llic Uiiiversily or C Y ~~L I S .He w i i s llic rccipient n C the 1994 Europc;iii Commimily "Marie-Citric" fellowship, ;md lic is iiclively pilrlicipating iii a number of European prqjecls related t n health klcmiitics.His current research iiitci inedical imaging, hiosignal analysis, x t ificial nciiriil nclworks, iind genetic algcjrilhins.lie has published (ivcr 40 re1crced ioiirniil and conference papers iii llicsc iircas.Dr. Pallichis liiis becii invoIvcd in nii-Iiidi;~na, in 1988, ;md the P1i.D. deuce in systems lhcory froni llic University of Lolidon in 1981.He rcceivcd the 1979 Williain Lincoln Shelley a W " I 1roiii (lie University r i f London for exccllcncc iii rcsearch, and ti Fulhright 1cllowship for col-Iiiborativc rcscarch iii Ihc USA in 1993.He was a Ipostdocloral Ccllirw a1 the University nf I .oiidnnfrom I 9X I tu 1983, taught iis a Icctum olcompulcr sciciiceiit lhc Higher Teclinical Instilulc i l l Cypriis frmii I985 tu 1990, and was a iprolessor of computer inform;ttion s y s ~c m s at the U nvcrsity o1Indian;ipolis lroiii I990 L o 1992.Since 1992, he has been with the Departinciil 01 Computer Science, University of Cyprus, as ii prolessor, where he served iis Interim Chair oC the Departmenl iiiilil 1994.Hc i s also the direclor o1Cmiipulationiil Intclligcnce Research ill the Cyprus Iiistilutc OT Neurology and Gcnelics.His resciirch i~iLercsls iiicliidc coinpulalinniil iiilelligencc, clccisioii suppnrl syslciiis, medicill ;ippications, and diagiioslic sys-Iciiis.Me has ptihlislied over 70 rclbrced jourtial aiid cuiil'crcncc papers on thcsc ar-.Dr yclliv, 'I\I\'II'ellow~~ftlrelE~.alld .' ., RCS, iind since 1992 lias been a tncinbcr of the International Commillec nT tlie IEEE-EMBS aiiiiiial conference.I l e i s ii mcmhcr 111 Ihc miiiislcrial commillcc h r cslablishing Ilic Inf~~rm~iiion Socicly in Cyprus, ;ind lie i s ;I p;"incr i n viirims I%r o ~~c i i i i Union limdcd [projects.SIi!f2nos Kol1iii.s was horii in Alliciis, Greece, in 1956.Hcoblained Iiis Dipluiiiii in elcctriciil ancl mcclianictil ciiginceriiig from the Na~iiinal Technical University 01 Athens (NI'UA) in 1979.(lie M.Sc.iii c o ~i r ~i i ~i i i i c a l i o i i cnginecritig from llic University 01M;inchestcr Instilulc of Scimil Tcchndogy in Eng and the P1i.D. in sign;il pro Ilic C~,inpLitcrSci~ncc Divisi I n 1982, lit: was givcn aCOMSOC Scholarship lioin (lie IEEE Coin~nu~iic:ition Society.Since 19x6, lie has been with the Departincnt [ i f Electrical iind Compulcr Engineering 0 1 NTUA, where Iic currctilly i s pri~Scssor and direclor of [lie lmage Proccssing and M u l l i m e d i a Lahoralory.Froiii 19x7 to 1988, hc w Visiting Itcscarch Scientist iti lhe Dcparliiiciit o1Electrical Eiigineering o f Colum-Ilia University i n New York, USA, on leave from NTUA.His rcszarch interests iiicludc n c u r ~I networks, liumaii coinpuler in~crac~ion, imagc/vidco processing i ~n d analysis, intclligctil ~i i u l ~i i n e d i a ~iicdic;il im;iging.Dr. Kolliashas published 40 papcrs in inlcriialioiial journals imd XI) in proceedings 01 internalion;il co~i~ercnccs.Since 1990, Iic has becn leading m(irc lliiiii 30 rcscwch prnjccts at Ihc Crcck :ind Europcati IcvcI.M'rriii Miiv~irwiii V m d i o i i was horn i t i Limassul, Cyprus, i n I95 I, Slic sludicd nictliciiic in tlic University o f Alliens Medical Schuiil, wlierc slic ohl;iincd Iier M.D. i n 1977.Slic then spccializcd iii Iiislopilhology in Sl.Savvas Anti-Cancer Hospital, Athciis, ;iod in 1983.Relween 1987.1991slic worked privately i n I,iiiiassiiI.Sincc 1991, she has workad i n the Hislopathology Dcpartmeiil of the Nicosia Gencral Hospital.She is it nieinher 01 the Hellenic Socicly cif Anatomic Palliology, lhe tlcllcnic Division of the Inlcrnalional Ac;ldcmy of Pathology, aiid also a Member of h e European Group for Rreast Canccr Screening.She is espec i n l l y i n t c r c s t c d i n b r c a s ~ and gynccological cancer.ArlriiiIou was horn in Cyprus in I950 iind altcnded llic Medical School of lhc University n1 Alhciis, where he receivcd his MD in 1976.He worked al St. Aiiargyri C;mccr Hospitd as a junior doclor Ilicii was prumutcd lo assishint d i utor.From 1979 lo 19x4, lie did his fellowship in iiilernal iiicdicine at tlic Red Cross tlospitd i l l Allieos.Between May iiiid Augusl of 1984 lie wiis on a scholarship for iiiediciil oncology a1 Roswcll Park Mcniorial Instilute i n Rufi'alo, New York.In 1985 hc rctorncd lo Cyprus mil workod privately imlil 1988, wlicii he was appointed consultant mcdiciil oncologist a i the Nicosia Gciicral Hospihil.In 1997, he was promotcd lo assisliinl dircclor of the Clinical Oncology Deparlinciil of Nicosi:i General Ilospilal.He i s alsu an Iiniio1.iiryc ~i i ~u l t a i i t OS St. Barlhloincw's Hospilal, London, and since 1992 a visiting prolcssor of medical oncology iit the Sheha Medictil Cenlcr (lcl-Hashumer Hospital) Israel.He has recently bccii alppoiriled licnd of tlic Breast and Gyneccological Cancer Clinic at lhc B m k of Cyprus Oocology Centre.H i s interests iiielude llic maiiagcinent oT breast and other lcinalc malignaiicics a s well its lhc role of genes in lainilial cancers.K y i i i c i i .~ C. Kyrincoii wiis born i n Nicosia, Cyprus, in 1954.H e received liis B.Sc. degrcc i n hi ocJic~ii i st r y lrom the Uiiivcrsily 01' London iii 1977 a n d his P1i.D. fvom King's College Medical School, London, iii 1982.During liis P1i.D. rese:ircIi.lic speci:ili/,cd iii Ihc use of liistiipathological ~ccliniqiies, including clcc~roii inicroscopy, Tor diagnostic as well its for resciirch npplicalions.In 1982 lie was appointed Icclurer ;It King's Col- Ihe Uiiivcrsily o f ~ 'IS.aiid