Texture Modeling

From Dahuawiki

Introduction

Study of texture is one of the fundamental topic in computer vision. Despite extensive works in this topic, the essential question what is texture is sitll open. The understanding of this question varies among different people, which results in different philosophy in describing textures.

Generally speaking, texture models can be classified into two categories:

Deterministic textures

use a set of placement rules of primitives to describe the composition of textures

Stochastic textures

consider textures as observations generated by underlying stochastic process

It seems that the machine learning and computer vision community tends to favor the latter in recent years since it has several advantages

• Easy to use - a good algorithm can be applied to a broad range of textures requiring little efforts from user
• Elegant - the theory of stochastic textures is rooted in statistics.
• Robust - it tolerates all kinds of uncertainty, noise, and deformation that are typical and reality owning to its stochastic nature.

The following discusses some representative works in texture modeling. (It will be updated as I read more papers)

Filtering Models

A widely used family of methods to model textures is based on filtering, which is generally based on the rationale that the visual characteristics of the textures can be effectively captured in the statistics or distributions of filter responses. The research in neurophysiology suggests that the human visual system decomposes the retinal image into sub-bands, which has close relation with filtering. Typical filter bands employed in vision literatures include Wavelet, Gabor filters, and Steerable filters, etc.

Traditionally, filter-based methods were mainly used for texture description, in which statistics or histograms of filter responses are extracted as features to represent the textures. Later, complete texture models for both analysis and synthesis were developed. Representative works include Zhu's FRAME model and pyramid-based models.

Discussions on related papers

• Statistical models based on filtering
  • Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling. S-C. Zhu, Y. Wu, and D. Mumford. (IJCV 1998).

    A generative texture model based on filters and maximum entropy principle.

• Pyramid based models
  • Pyramid-Based Texture Analysis/Synthesis. D. J. Heeger and J. R. Bergen. SIGGRAPH (1995).

    A pioneer work in pyramid-based texture modeling that performs texture matching by aligning the histograms of filter responses.

  • A Non-parametric Multi-Scale Statistical Model for Natural Images. J. S. De Bonet and P. Viola. NIPS (1997).

    A pyramid-based statistical model of textures with cross-scale dependency explicitly taken into account with an Markov tree over the parent structure.

  • Multiresolution Sampling Procedure for Analysis and Synthesis of Textured Images. J. S. De Bonet. SIGGRAPH (1995).

    An algorithm to analyze and synthesize textures using non-parametric sampling based on the statistical model presented in Bonet's NIPS'97 paper.

Texton Models

Recently, texton models become increasingly popular in vision community due to its uncomparable expressive power. The texton models consider that textures are formed by composition of a set of units called textons. This terminology is analogous to electron or atom in physics. The spatial configuration or statistical distributions of the textons characterize textures. Though widely used, the word texton is actually not clearly defined. The understanding of what are textons vary among different researchers.

Generally speaking, texton models fall in three different categories

Clusters of local patterns

The underlying rationale is that typical textured images present regular local patterns, which form clustered structures in their space. These methods often have two stages. Build a texton codebook (dictionary) by learning the local pattern clusters (k-means is often used here) from a large number of local patches extracted from images. After the codebook is constructed, it can be applied to describe textures by the distribution of the textons. Owning to its simplicity, this method is widely employed in various domains.

Localized basis

This approach sees each textured image as superposition of multiple localized bases. The bases are typically learned from training set, which are then used to solve the coefficient for testing images. The coefficients are considered to characterize some intrinsic features of textures. A representative work here is sparse coding.

Generative models

They directly model the formation of textures, in which textons are considered as primitive units to construct the textured image. These models are often based on generative probabilistic models in which textons serve as intermediate layer to connect low level pixels and high level models. In addition, sampling procedure may be required in analysis and synthesis. Though theoretically appealing, they are not used as widely as the other approaches due to the difficulty of implementation and relatively lower run-time efficiency.

Discussions on related papers

• Methods based on clustering/quantization
  • Representing and Recognizing the Visual Appearance of Materials Using Three-dimensional Textons T. Leung and J. Malik. IJCV (2001).

    Revisit the concept of texton - develop a texture model based on texton vocabulary and histograms, with a simple extension to model 3D textons.

• Methods based on localized basis
  • Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1? B.A. Olshausen and D.J. Field. Vision Research (1997).

    An image encoding method based on a linear generative model with sparse prior.

• Sophisticated generative models
  • What are Textons? S-C. Zhu, C-E. Guo, Y-Z. Wang, and Z-J. Xu. IJCV (2005).

    A sophisticated generative model of textures with three levels: image - bases - textons. A milestone in texton modeling.