KeyValueDatastore

Datastore for key-value pair data

expand all in page

Description

KeyValueDatastore objects are associated with files containing key-value pair data that are outputs of or inputs to mapreduce. Use the KeyValueDatastore properties to specify how you want to access the data. Use dot notation to view or modify a particular property of a KeyValueDatastore object:

ds = datastore('mapredout.mat');
ds.ReadSize = 20;

You also can specify the value of KeyValueDatastore properties using name-value pair arguments when you create a datastore using the datastore function:

ds = datastore('mapredout.mat','ReadSize',20);

Create Object

Create KeyValueDatastore objects using the datastore function.

Properties

expand all

Files included in the datastore, specified as an n-by-1 cell array of character vectors, where each character vector is a full path to a file. These are the files defined by the location argument to the datastore function.

The files must be either MAT-files or Sequence files generated by the mapreduce function. The files must be in a local file system or in a network file system. Sequence files also can be in HDFS™.

Example: {'C:\dir\data\file1.mat';'C:\dir\data\file2.mat'}

File type, specified as either 'mat' for MAT-files or 'seq' for sequence files. By default, the output of mapreduce running against Hadoop® is a datastore containing sequence files. By default, the output of all other mapreduce operations is a datastore containing MAT-files.

Maximum number of key-value pairs to read in a call to the read or preview functions, specified as a positive integer.

Object Functions

hasdata Determine if data is available to read
numpartitions Number of partitions
partition Partition a datastore
preview Return subset of data from datastore
read Read data in datastore
readall Read all data in datastore
reset Reset datastore to initial state

Examples

expand all

Create a datastore from the sample file, mapredout.mat, which is an output file of the mapreduce function.

ds = datastore('mapredout.mat')
ds = 
  KeyValueDatastore with properties:

       Files: {
              ' ...\matlab\toolbox\matlab\demos\mapredout.mat'
              }
    ReadSize: 1 key-value pairs
    FileType: 'mat'

Set the ReadSize property to 8 so that each call to read reads at most 8 key-value pairs.

ds.ReadSize = 8
ds = 
  KeyValueDatastore with properties:

       Files: {
              ' ...\matlab\toolbox\matlab\demos\mapredout.mat'
              }
    ReadSize: 8 key-value pairs
    FileType: 'mat'

Read 8 key-value pairs at a time using the read function in a while loop. The loop executes until there is no more data available to read and hasdata(ds) returns false.

while hasdata(ds)
    T = read(ds);
end

Show the last set of key-value pairs read.

T
T = 

    Key     Value 
    ____    ______

    'OO'    [3090]
    'TZ'    [ 216]
    'XE'    [2357]
    '9E'    [ 521]
    'YV'    [ 849]

Related Examples

See Also

|

Introduced in R2014b

Was this topic helpful?
MATLAB Documentation
Other Documentation
Support