Classifiers¶
SDR Classifier¶
Implementation of a SDR classifier.
The SDR classifier takes the form of a single layer classification network that takes SDRs as input and outputs a predicted distribution of classes.

class
nupic.algorithms.sdr_classifier.
SDRClassifier
(steps=[1], alpha=0.001, actValueAlpha=0.3, verbosity=0)¶ Bases:
nupic.serializable.Serializable
The SDR Classifier accepts a binary input pattern from the level below (the “activationPattern”) and information from the sensor and encoders (the “classification”) describing the true (target) input.
The SDR classifier maps input patterns to class labels. There are as many output units as the number of class labels or buckets (in the case of scalar encoders). The output is a probabilistic distribution over all class labels.
During inference, the output is calculated by first doing a weighted summation of all the inputs, and then perform a softmax nonlinear function to get the predicted distribution of class labels
During learning, the connection weights between input units and output units are adjusted to maximize the likelihood of the model
Example Usage:
c = SDRClassifier(steps=[1], alpha=0.1, actValueAlpha=0.1, verbosity=0) # learning c.compute(recordNum=0, patternNZ=[1, 5, 9], classification={"bucketIdx": 4, "actValue": 34.7}, learn=True, infer=False) # inference result = c.compute(recordNum=1, patternNZ=[1, 5, 9], classification={"bucketIdx": 4, "actValue": 34.7}, learn=False, infer=True) # Print the top three predictions for 1 steps out. topPredictions = sorted(zip(result[1], result["actualValues"]), reverse=True)[:3] for probability, value in topPredictions: print "Prediction of {} has probability of {}.".format(value, probability*100.0)
References:
 Alex Graves. Supervised Sequence Labeling with Recurrent Neural Networks, PhD Thesis, 2008
 J. S. Bridle. Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition
 In F. FoglemanSoulie and J.Herault, editors, Neurocomputing: Algorithms, Architectures and Applications, pp 227236, SpringerVerlag, 1990
Parameters:  steps – (list) Sequence of the different steps of multistep predictions to learn
 alpha – (float) The alpha used to adapt the weight matrix during learning. A larger alpha results in faster adaptation to the data.
 actValueAlpha – (float) Used to track the actual value within each bucket. A lower actValueAlpha results in longer term memory
 verbosity – (int) verbosity level, can be 0, 1, or 2
Raises: (ValueError) when record number does not increase monotonically.

compute
(recordNum, patternNZ, classification, learn, infer)¶ Process one input sample.
This method is called by outer loop code outside the nupicengine. We use this instead of the nupic engine compute() because our inputs and outputs aren’t fixed size vectors of reals.
Parameters:  recordNum – Record number of this input pattern. Record numbers normally increase sequentially by 1 each time unless there are missing records in the dataset. Knowing this information insures that we don’t get confused by missing records.
 patternNZ – List of the active indices from the output below. When the input is from TemporalMemory, this list should be the indices of the active cells.
 classification –
Dict of the classification information where:
 bucketIdx: list of indices of the encoder bucket
 actValue: list of actual values going into the encoder
Classification could be None for inference mode.
 learn – (bool) if true, learn this sample
 infer – (bool) if true, perform inference
Returns: Dict containing inference results, there is one entry for each step in self.steps, where the key is the number of steps, and the value is an array containing the relative likelihood for each bucketIdx starting from bucketIdx 0.
There is also an entry containing the average actual value to use for each bucket. The key is ‘actualValues’.
for example:
{1 : [0.1, 0.3, 0.2, 0.7], 4 : [0.2, 0.4, 0.3, 0.5], 'actualValues': [1.5, 3,5, 5,5, 7.6], }

infer
(patternNZ, actValueList)¶ Return the inference value from one input sample. The actual learning happens in compute().
Parameters:  patternNZ – list of the active indices from the output below
 classification – dict of the classification information: bucketIdx: index of the encoder bucket actValue: actual value going into the encoder
Returns: dict containing inference results, one entry for each step in self.steps. The key is the number of steps, the value is an array containing the relative likelihood for each bucketIdx starting from bucketIdx 0.
for example:
{'actualValues': [0.0, 1.0, 2.0, 3.0] 1 : [0.1, 0.3, 0.2, 0.7] 4 : [0.2, 0.4, 0.3, 0.5]}

inferSingleStep
(patternNZ, weightMatrix)¶ Perform inference for a single step. Given an SDR input and a weight matrix, return a predicted distribution.
Parameters:  patternNZ – list of the active indices from the output below
 weightMatrix – numpy array of the weight matrix
Returns: numpy array of the predicted class label distribution

class
nupic.algorithms.sdr_classifier_factory.
SDRClassifierFactory
¶ Factory for instantiating SDR classifiers.

static
create
(*args, **kwargs)¶ Create a SDR classifier factory. The implementation of the SDR Classifier can be specified with the “implementation” keyword argument.
 The SDRClassifierFactory uses the implementation as specified in
 Default NuPIC Configuration.

static
KNN Classifier¶

class
nupic.algorithms.knn_classifier.
KNNClassifier
(k=1, exact=False, distanceNorm=2.0, distanceMethod='norm', distThreshold=0, doBinarization=False, binarizationThreshold=0.5, useSparseMemory=True, sparseThreshold=0.1, relativeThreshold=False, numWinners=0, numSVDSamples=0, numSVDDims=None, fractionOfMax=None, verbosity=0, maxStoredPatterns=1, replaceDuplicates=False, cellsPerCol=0, minSparsity=0.0)¶ Bases:
nupic.serializable.Serializable
This class implements NuPIC’s k Nearest Neighbor Classifier. KNN is very useful as a basic classifier for many situations. This implementation contains many enhancements that are useful for HTM experiments. These enhancements include an optimized C++ class for sparse vectors, support for continuous online learning, support for various distance methods (including Lpnorm and raw overlap), support for performing SVD on the input vectors (very useful for large vectors), support for a fixedsize KNN, and a mechanism to store custom ID’s for each vector.
Parameters:  k – (int) The number of nearest neighbors used in the classification of patterns. Must be odd.
 exact – (boolean) If true, patterns must match exactly when assigning class labels
 distanceNorm – (int) When distance method is “norm”, this specifies the p value of the Lpnorm
 distanceMethod –
(string) The method used to compute distance between input patterns and prototype patterns. The possible options are:
norm
: When distanceNorm is 2, this is the euclidean distance, When distanceNorm is 1, this is the manhattan distance In general: sum(abs(xproto) ^ distanceNorm) ^ (1/distanceNorm) The distances are normalized such that farthest prototype from a given input is 1.0.
rawOverlap
: Only appropriate when inputs are binary. This computes: (width of the input)  (# bits of overlap between input and prototype).
pctOverlapOfInput
: Only appropriate for binary inputs. This computes 1.0  (# bits overlap between input and prototype) /
 (# ON bits in input)
pctOverlapOfProto
: Only appropriate for binary inputs. This computes 1.0  (# bits overlap between input and prototype) /
 (# ON bits in prototype)
pctOverlapOfLarger
: Only appropriate for binary inputs. This computes 1.0  (# bits overlap between input and prototype) /
 max(# ON bits in input, # ON bits in prototype)
 distThreshold – (float) A threshold on the distance between learned patterns and a new pattern proposed to be learned. The distance must be greater than this threshold in order for the new pattern to be added to the classifier’s memory.
 doBinarization – (boolean) If True, then scalar inputs will be binarized.
 binarizationThreshold – (float) If doBinarization is True, this specifies the threshold for the binarization of inputs
 useSparseMemory – (boolean) If True, classifier will use a sparse memory matrix
 sparseThreshold – (float) If useSparseMemory is True, input variables whose absolute values are less than this threshold will be stored as zero
 relativeThreshold – (boolean) Flag specifying whether to multiply sparseThreshold by max value in input
 numWinners – (int) Number of elements of the input that are stored. If 0, all elements are stored
 numSVDSamples – (int) Number of samples the must occur before a SVD (Singular Value Decomposition) transformation will be performed. If 0, the transformation will never be performed
 numSVDDims – (string) Controls dimensions kept after SVD transformation. If “adaptive”, the number is chosen automatically
 fractionOfMax – (float) If numSVDDims is “adaptive”, this controls the smallest singular value that is retained as a fraction of the largest singular value
 verbosity – (int) Console verbosity level where 0 is no output and larger integers provide increasing levels of verbosity
 maxStoredPatterns – (int) Limits the maximum number of the training patterns stored. When KNN learns in a fixed capacity mode, the unused patterns are deleted once the number of stored patterns is greater than maxStoredPatterns. A value of 1 is no limit
 replaceDuplicates – (bool) A boolean flag that determines whether, during learning, the classifier replaces duplicates that match exactly, even if distThreshold is 0. Should be True for online learning
 cellsPerCol – (int) If >= 1, input is assumed to be organized into columns, in the same manner as the temporal memory AND whenever a new prototype is stored, only the start cell (first cell) is stored in any bursting column
 minSparsity – (float) If useSparseMemory is set, only vectors with sparsity >= minSparsity will be stored during learning. A value of 0.0 implies all vectors will be stored. A value of 0.1 implies only vectors with at least 10% sparsity will be stored

clear
()¶ Clears the state of the KNNClassifier.

closestOtherTrainingPattern
(inputPattern, cat)¶ Return the closest training pattern that is not of the given category “cat”.
Parameters:  inputPattern – The pattern whose closest neighbor is sought
 cat – Training patterns of this category will be ignored no matter their distance to inputPattern
Returns: A dense version of the closest training pattern, or None if no such patterns exist

closestTrainingPattern
(inputPattern, cat)¶ Returns the closest training pattern to inputPattern that belongs to category “cat”.
Parameters:  inputPattern – The pattern whose closest neighbor is sought
 cat – The required category of closest neighbor
Returns: A dense version of the closest training pattern, or None if no such patterns exist

computeSVD
(numSVDSamples=0, finalize=True)¶ Compute the singular value decomposition (SVD). The SVD is a factorization of a real or complex matrix. It factors the matrix a as u * np.diag(s) * v, where u and v are unitary and s is a 1d array of a‘s singular values.
Reason for computing the SVD:
There are cases where you want to feed a lot of vectors to the KNNClassifier. However, this can be slow. You can speed up training by (1) computing the SVD of the input patterns which will give you the eigenvectors, (2) only keeping a fraction of the eigenvectors, and (3) projecting the input patterns onto the remaining eigenvectors.
Note that all input patterns are projected onto the eigenvectors in the same fashion. Keeping only the highest eigenvectors increases training performance since it reduces the dimensionality of the input.
Parameters:  numSVDSamples – (int) the number of samples to use for the SVD computation.
 finalize – (bool) whether to apply SVD to the input patterns.
Returns: (array) The singular values for every matrix, sorted in descending order.

doIteration
()¶ Utility method to increment the iteration index. Intended for models that don’t learn each timestep.

finishLearning
()¶ Used for batch scenarios. This method needs to be called between learning and inference.

getAdaptiveSVDDims
(singularValues, fractionOfMax=0.001)¶ Compute the number of eigenvectors (singularValues) to keep.
Parameters:  singularValues –
 fractionOfMax –
Returns:

getClosest
(inputPattern, topKCategories=3)¶ Returns the index of the pattern that is closest to inputPattern, the distances of all patterns to inputPattern, and the indices of the k closest categories.

getDistances
(inputPattern)¶ Return the distances between the input pattern and all other stored patterns.
Parameters: inputPattern – pattern to check distance with Returns: (distances, categories) numpy arrays of the same length.  overlaps: an integer overlap amount for each category  categories: category index for each element of distances

getNumPartitionIds
()¶ Returns: the number of unique partition Ids stored.

getOverlaps
(inputPattern)¶ Return the degree of overlap between an input pattern and each category stored in the classifier. The overlap is computed by computing:
logical_and(inputPattern != 0, trainingPattern != 0).sum()
Parameters: inputPattern – pattern to check overlap of Returns: (overlaps, categories) Two numpy arrays of the same length, where:  overlaps: an integer overlap amount for each category
 categories: category index for each element of overlaps

getPartitionId
(i)¶ Gets the partition id given an index.
Parameters: i – index of partition Returns: the partition id associated with pattern i. Returns None if no id is associated with it.

getPartitionIdKeys
()¶ Returns: a list containing unique (nonNone) partition Ids (just the keys)

getPartitionIdList
()¶ Returns: a list of complete partition id objects

getPattern
(idx, sparseBinaryForm=False, cat=None)¶ Gets a training pattern either by index or category number.
Parameters:  idx – Index of the training pattern
 sparseBinaryForm – If true, returns a list of the indices of the nonzero bits in the training pattern
 cat – If not None, get the first pattern belonging to category cat. If this is specified, idx must be None.
Returns: The training pattern with specified index

getPatternIndicesWithPartitionId
(partitionId)¶ Returns: a list of pattern indices corresponding to this partitionId. Return an empty list if there are none.

infer
(inputPattern, computeScores=True, overCategories=True, partitionId=None)¶ Finds the category that best matches the input pattern. Returns the winning category index as well as a distribution over all categories.
Parameters:  inputPattern – (list or array) The pattern to be classified. This must be a dense representation of the array (e.g. [0, 0, 1, 1, 0, 1]).
 computeScores – NO EFFECT
 overCategories – NO EFFECT
 partitionId – (int) If provided, all training vectors with partitionId equal to that of the input pattern are ignored. For example, this may be used to perform kfold cross validation without repopulating the classifier. First partition all the data into k equal partitions numbered 0, 1, 2, ... and then call learn() for each vector passing in its partitionId. Then, during inference, by passing in the partition ID in the call to infer(), all other vectors with the same partitionId are ignored simulating the effect of repopulating the classifier while ommitting the training vectors in the same partition.
Returns: 4tuple with these keys:
winner
: The category with the greatest number of nearest neighbors within the kth nearest neighbors. If the inferenceResult contains no neighbors, the value of winner is None. This can happen, for example, in cases of exact matching, if there are no stored vectors, or if minSparsity is not met.
inferenceResult
: A list of length numCategories, each entry contains the number of neighbors within the top k neighbors that are in that category.
dist
: A list of length numPrototypes. Each entry is the distance from the unknown to that prototype. All distances are between 0.0 and 1.0.
categoryDist
: A list of length numCategories. Each entry is the distance from the unknown to the nearest prototype of that category. All distances are between 0 and 1.0.

learn
(inputPattern, inputCategory, partitionId=None, isSparse=0, rowID=None)¶ Train the classifier to associate specified input pattern with a particular category.
Parameters:  inputPattern – (list) The pattern to be assigned a category. If isSparse is 0, this should be a dense array (both ON and OFF bits present). Otherwise, if isSparse > 0, this should be a list of the indices of the nonzero bits in sorted order
 inputCategory – (int) The category to be associated to the training pattern
 partitionId – (int) partitionID allows you to associate an id with each input vector. It can be used to associate input patterns stored in the classifier with an external id. This can be useful for debugging or visualizing. Another use case is to ignore vectors with a specific id during inference (see description of infer() for details). There can be at most one partitionId per stored pattern (i.e. if two patterns are within distThreshold, only the first partitionId will be stored). This is an optional parameter.
 isSparse – (int) 0 if the input pattern is a dense representation. When the input pattern is a list of nonzero indices, then isSparse is the number of total bits (n). E.g. for the dense array [0, 1, 1, 0, 0, 1], isSparse should be 0. For the equivalent sparse representation [1, 2, 5] (which specifies the indices of active bits), isSparse should be 6, which is the total number of bits in the input space.
 rowID – (int) UNKNOWN
Returns: The number of patterns currently stored in the classifier

prototypeSetCategory
(idToCategorize, newCategory)¶ Allows ids to be assigned a category and subsequently enables users to use:
removeCategory()
closestTrainingPattern()
closestOtherTrainingPattern()

remapCategories
(mapping)¶ Change the category indices.
Used by the Network Builder to keep the category indices in sync with the ImageSensor categoryInfo when the user renames or removes categories.
Parameters: mapping – List of new category indices. For example, mapping=[2,0,1] would change all vectors of category 0 to be category 2, category 1 to 0, and category 2 to 1

removeCategory
(categoryToRemove)¶ There are two caveats. First, this is a potentially slow operation. Second, pattern indices will shift if patterns before them are removed.
Parameters: categoryToRemove – Category label to remove

removeIds
(idsToRemove)¶ There are two caveats. First, this is a potentially slow operation. Second, pattern indices will shift if patterns before them are removed.
Parameters: idsToRemove – A list of row indices to remove.

setCategoryOfVectors
(vectorIndices, categoryIndices)¶ Change the category associated with this vector(s).
Used by the Network Builder to move vectors between categories, to enable categories, and to invalidate vectors by setting the category to 1.
Parameters:  vectorIndices – Single index or list of indices
 categoryIndices – Single index or list of indices. Can also be a single index when vectorIndices is a list, in which case the same category will be used for all vectors