Assistive Cloth Pattern Recognition for Visually Impaired People

In this paper, we aim at a practical system, magic closet, for automatic occasion-orientedclothing recommendation. Given a user-input occasion, e.g., wedding, shopping or dating, magic closet intelligently suggests the most suitable clothing from the user’s own clothing photo album, or automatically pairs the userspecified reference clothing (upper-body or lower-body)with the most suitable one from online shops. Choosing clothes with complex patterns and colors is a challenging task for visually impaired people. Automatic clothing pattern recognitionis also a challenging research problem due to rotation,scaling, illumination, and especially large intra class pattern variations. We have developed a camera-based prototype system that recognizes clothing patterns in four categories (plaid, striped, pattern less, and irregular) and identifies 11 clothing colors. The system integrates a camera, a microphone, a computer, and a Bluetooth earpiece for audio description of clothing patterns and colors. A camera mounted upon a pair of sunglasses is used to capture clothing images. The clothing patterns and colors are described to blind users verbally. This system can be controlled by speech input through microphone. To recognize clothing patterns, we propose a novel Radon Signature descriptor and schemata extract statistical properties from wavelet sub bands to capture global features of clothing patterns. More specifically, the clothing attributes are treated as latent variables in ourproposed latent Support Vector Machine (SVM) based recommendation model. The wearing properly criterion is described in the model through a feature-occasion potential and an attribute-occasion potential, while the wearing aesthetically criterion is expressed by an attribute- attribute potential. The prototype was also used by ten visually impaired participants. Most thought such a system would support more independence in their daily life but they also made suggestions for improvements