All Classes Interface Summary Class Summary Enum Summary Exception Summary
| Class |
Description |
| ABitmap |
|
| ABooleanArray |
|
| ACachingMBDictionary |
|
| ACLAScheme |
|
| AColGroup |
Abstract Class that is the lowest class type for the Compression framework.
|
| AColGroup.CompressionType |
Public super types of compression ColGroups supported
|
| AColGroupCompressed |
Base class for column groups encoded Encoded in a compressed manner.
|
| AColGroupOffset |
Base class for column groups encoded with various types of bitmap encoding.
|
| AColGroupValue |
|
| AColIndex |
|
| AColumnCoCoder |
Main abstract class for the co-coding of columns to combine different compression statistics and calculate the
combinations of columns
|
| AComEst |
Main abstract class for estimating size of compressions on columns.
|
| ACompressedArray<T> |
A Compressed Array, in general does not allow us to set or modify the array.
|
| ACostEstimate |
A cost estimator interface.
|
| ACount<T> |
|
| ACount.DArrCounts |
|
| ACount.DCounts |
|
| ACountHashMap<T> |
|
| ADictBasedColGroup |
|
| ADictionary |
This dictionary class aims to encapsulate the storage and operations over unique tuple values of a column group.
|
| AEncode |
|
| AFederatedPlanner |
|
| AggBinaryOp |
|
| AggBinaryOp.MMultMethod |
|
| AggBinaryOp.SparkAggType |
|
| AggregateBinaryCPInstruction |
|
| AggregateBinaryFEDInstruction |
|
| AggregateBinaryGPUInstruction |
|
| AggregateBinaryOperator |
|
| AggregateBinarySPInstruction |
Class to group the different MM SPInstructions together.
|
| AggregateDropCorrectionFunction |
|
| AggregateOperator |
|
| AggregateTernaryCPInstruction |
|
| AggregateTernaryFEDInstruction |
|
| AggregateTernaryOperator |
|
| AggregateTernarySPInstruction |
|
| AggregateUnaryCPInstruction |
|
| AggregateUnaryCPInstruction.AUType |
|
| AggregateUnaryFEDInstruction |
|
| AggregateUnaryGPUInstruction |
Implements aggregate unary instructions for CUDA
|
| AggregateUnaryOperator |
|
| AggregateUnarySketchSPInstruction |
|
| AggregateUnarySPInstruction |
|
| AggregateUnarySPInstruction.RDDUAggFunction2 |
Similar to RDDUAggFunction but single output block.
|
| AggregateUnarySPInstruction.RDDUTensorAggFunction2 |
Similar to RDDUAggFunction but single output block.
|
| AggUnaryOp |
|
| AIdentityDictionary |
|
| AInsertionSorter |
This abstract class is for sorting the IntArrayList entries efficiently for SDC Column Groups construction.
|
| AIterator |
Iterator interface, that returns a iterator of the indexes while also maintaining a data index.
|
| AMapToData |
This Class's job is to link into the dictionary entries for column groups.
|
| AMorphingMMColGroup |
Abstract class for column group types that do not perform matrix Multiplication, and decompression for performance
reasons but instead transforms into another type of column group type to perform that operation.
|
| And |
|
| AOffset |
Offset list encoder abstract class.
|
| AOffset.OffsetSliceInfo |
Offset slice info containing the start and end index an offset that contains the slice, and an new AOffset
containing only the sliced elements
|
| AOffsetByte |
|
| AOffsetIterator |
Iterator interface that only iterate through offsets without considering data index.
|
| AOffsetsGroup |
|
| Append |
|
| AppendCPInstruction |
|
| AppendCPInstruction.AppendType |
|
| AppendFEDInstruction |
|
| AppendG |
TODO Additional compiler enhancements:
1) Partial Shuffle Elimination - Any full or aligned blocks could be directly output from the mappers
to the result index.
|
| AppendGAlignedSP |
|
| AppendGAlignedSPInstruction |
|
| AppendGAlignedSPInstruction.ShiftColumnIndex |
|
| AppendGSPInstruction |
|
| AppendGSPInstruction.MergeWithShiftedBlocks |
|
| AppendGSPInstruction.ShiftMatrix |
|
| AppendM |
|
| AppendM.CacheType |
|
| AppendMSPInstruction |
|
| AppendR |
|
| AppendRSPInstruction |
|
| AppendSPInstruction |
Class to group the different append SPInstructions together.
|
| APreAgg |
Abstract class for all the column groups that use preAggregation for Left matrix multiplications.
|
| ArithmeticBinaryGPUInstruction |
|
| Array<T> |
Generic, resizable native arrays for the internal representation of the columns in the FrameBlock.
|
| ArrayCompressionStatistics |
|
| ArrayFactory |
|
| ArrayFactory.FrameArrayType |
|
| ArrayIndex |
|
| ArrayWrapper |
|
| ASDC |
Column group that sparsely encodes the dictionary values.
|
| ASDCZero |
|
| AssignmentStatement |
|
| AutoDiff |
|
| AWTreeNode |
A workload tree is a compact representation of the operations on a matrix and derived intermediates, including the
basic control structure and inlined functions as well as links to categories.
|
| AWTreeNode.WTNodeType |
|
| BalanceToAvgFederatedScheme |
Balance to Avg Federated scheme
When the parameter server runs in federated mode it cannot pull in the data which is already on the workers.
|
| BaseModel |
|
| BasicProgramBlock |
|
| BasicTensorBlock |
|
| Binary |
|
| Binary |
|
| Binary |
Lop to perform binary operation.
|
| BinaryBlockSerialization |
This custom serialization class can be used via
job.set(MRConfigurationNames.IO_SERIALIZATIONS, "org.apache.sysds.runtime.io.BinaryBlockSerialization");
|
| BinaryBlockSerialization.MatrixBlockDeserializer |
|
| BinaryBlockSerialization.MatrixBlockSerializer |
|
| BinaryBlockToTextCellConverter |
|
| BinaryCPInstruction |
|
| BinaryExpression |
|
| BinaryFEDInstruction |
|
| BinaryFrameFrameCPInstruction |
|
| BinaryFrameFrameSPInstruction |
|
| BinaryFrameMatrixCPInstruction |
|
| BinaryFrameMatrixSPInstruction |
|
| BinaryM |
Lop to perform binary operation.
|
| BinaryM.VectorType |
|
| BinaryMatrixBVectorSPInstruction |
|
| BinaryMatrixMatrixCPInstruction |
|
| BinaryMatrixMatrixFEDInstruction |
|
| BinaryMatrixMatrixSPInstruction |
|
| BinaryMatrixScalarCPInstruction |
|
| BinaryMatrixScalarFEDInstruction |
|
| BinaryMatrixScalarSPInstruction |
|
| BinaryOp |
Binary (cell operations): aij + bij
Properties:
Symbol: *, -, +, ...
|
| BinaryOp.AppendMethod |
|
| BinaryOp.MMBinaryMethod |
|
| BinaryOperator |
BinaryOperator class for operations that have two inputs.
|
| BinaryScalar |
Lop to perform binary scalar operations.
|
| BinaryScalarScalarCPInstruction |
|
| BinarySPInstruction |
|
| BinaryTensorTensorBroadcastSPInstruction |
|
| BinaryTensorTensorCPInstruction |
|
| BinaryTensorTensorSPInstruction |
|
| BinaryUAggChain |
|
| BinUaggChainSPInstruction |
Instruction that performs
res = X / rowsum(x)
|
| BinUaggChainSPInstruction.RDDBinUaggChainFunction |
|
| Bitmap |
Uncompressed representation of one or more columns in bitmap format.
|
| BitmapEncoder |
Static functions for extracting bitmaps from a MatrixBlock.
|
| BitMapValueCombiner |
|
| BitSetArray |
|
| BitwAnd |
|
| BitwOr |
|
| BitwShiftL |
|
| BitwShiftR |
|
| BitwXor |
|
| Block |
|
| BlockPartitioner |
Default partitioner used for all binary block rdd operations in order
to enable sufficient local aggregation independent of the aggregation
direction (row/col-wise).
|
| BooleanArray |
|
| BooleanArray32 |
|
| BooleanExpression |
|
| BooleanIdentifier |
|
| BooleanObject |
|
| BroadcastCPInstruction |
|
| BroadcastObject<T extends CacheBlock<?>> |
|
| Builtin |
Class with pre-defined set of objects.
|
| Builtin.BuiltinCode |
|
| BuiltinBinaryGPUInstruction |
|
| BuiltinConstant |
These are the builtin constants
|
| BuiltinFunctionExpression |
|
| BuiltinNaryCPInstruction |
Instruction to handle a variable number of input operands.
|
| BuiltinNarySPInstruction |
|
| BuiltinNarySPInstruction.PadBlocksFunction |
|
| Builtins |
Enum to represent all builtin functions in the default name space.
|
| BuiltinUnaryGPUInstruction |
|
| ByteBuffer |
Wrapper for WriteBuffer byte array per matrix/frame in order to
support matrix/frame serialization outside global lock.
|
| ByteBufferDataInput |
|
| ByteClassLoader |
|
| CacheableData<T extends CacheBlock<?>> |
Each object of this class is a cache envelope for some large piece of data
called "cache block".
|
| CacheableData.CacheStatus |
Defines all possible cache status types for a data blob.
|
| CacheBlock<T> |
Interface for all blocks handled by lazy write buffer.
|
| CacheBlockFactory |
Factory to create instances of matrix/frame blocks given
internal codes.
|
| CacheDataInput |
|
| CacheDataOutput |
Custom DataOutput to serialize directly into the given byte array.
|
| CachedReuseVariables |
|
| CacheEvictionQueue |
|
| CacheMaintenanceService |
|
| CacheStatistics |
This singleton provides basic caching statistics in CP.
|
| CastFEDInstruction |
|
| CastSPInstruction |
|
| CentralMoment |
Lop to perform cross product operation
|
| CentralMomentCPInstruction |
|
| CentralMomentFEDInstruction |
|
| CentralMomentSPInstruction |
|
| CharArray |
|
| Checkpoint |
Lop for checkpoint operations.
|
| CheckpointSPInstruction |
|
| CiphertextMatrix |
This class abstracts over an encrypted matrix of ciphertexts.
|
| CLALibAggTernaryOp |
|
| CLALibBinaryCellOp |
|
| CLALibBinCompress |
|
| CLALibCBind |
|
| CLALibCMOps |
|
| CLALibCombineGroups |
Library functions to combine column groups inside a compressed matrix.
|
| CLALibCompAgg |
|
| CLALibDecompress |
Library to decompress a list of column groups into a matrix.
|
| CLALibLeftMultBy |
|
| CLALibMatrixMult |
|
| CLALibMerge |
|
| CLALibMMChain |
Support compressed MM chain operation to fuse the following cases :
|
| CLALibReorg |
|
| CLALibReplace |
|
| CLALibReshape |
|
| CLALibRexpand |
|
| CLALibRightMultBy |
|
| CLALibScalar |
|
| CLALibScheme |
|
| CLALibSelectionMult |
This lib is responsible for selecting and extracting specific rows or columns from a compressed matrix.
|
| CLALibSeparator |
The job of this Lib is to separate and combine back a list of column groups from its dictionaries, and vice versa
combine back together dictionaries with their respective column groups
|
| CLALibSeparator.SeparatedGroups |
|
| CLALibSlice |
|
| CLALibSquash |
Squash or recompress is processing each column group and trying to find a better compressed representation for each.
|
| CLALibStack |
|
| CLALibTernaryOp |
|
| CLALibTSMM |
|
| CLALibUnary |
|
| CLALibUtils |
|
| CloudInstance |
This class describes the configurations of a single VM instance.
|
| CloudUtils |
Class providing static utilities for cloud related operations.
|
| CloudUtils.CloudProvider |
|
| CloudUtils.InstanceFamily |
|
| CloudUtils.InstanceSize |
|
| CM |
GENERAL NOTE:
* 05/28/2014: We decided to do handle weights consistently to SPSS in an operation-specific manner,
i.e., we (1) round instead of casting where required (e.g.
|
| CM_COV_Object |
|
| CMOperator |
|
| CMOperator.AggregateOperationTypes |
|
| CNode |
|
| CNodeBinary |
|
| CNodeBinary.BinType |
|
| CNodeCell |
|
| CNodeData |
|
| CNodeMultiAgg |
|
| CNodeNary |
|
| CNodeNary.NaryType |
|
| CNodeOuterProduct |
|
| CNodeRow |
|
| CNodeTernary |
|
| CNodeTernary.TernaryType |
|
| CNodeTpl |
|
| CNodeUnary |
|
| CNodeUnary.UnaryType |
|
| CoCodeBinPacking |
Column group partitioning with bin packing heuristic.
|
| CoCodeGreedy |
|
| CoCodeHybrid |
This cocode strategy starts combines the use of CoCodePriorityQue and CoCodeGreedy.
|
| CoCodePriorityQue |
Column group partitioning by number of distinct items estimated.
|
| CoCoderFactory |
|
| CoCoderFactory.PartitionerType |
The Valid coCoding techniques
|
| CoCodeStatic |
Column group co coding with static distribution heuristic.
|
| CodegenStatistics |
|
| CodeGenTrie |
|
| CodeGenTrieNode |
|
| CodeGenTrieNode.NodeType |
|
| CodegenUtils |
|
| CodeTemplate |
|
| COGByteReader |
This class is used by the COGReader to read bytes from a BufferedInputStream.
|
| COGCompressionUtils |
|
| COGHeader |
Represents a header for a COG file.
|
| COGProperties |
Properties of a COG file that are useful for reading the file.
|
| ColGroupConst |
|
| ColGroupDDC |
Class to encapsulate information about a column group that is encoded with dense dictionary encoding (DDC).
|
| ColGroupDDCFOR |
Class to encapsulate information about a column group that is encoded with dense dictionary encoding (DDC).
|
| ColGroupDeltaDDC |
Class to encapsulate information about a column group that is first delta encoded then encoded with dense dictionary
encoding (DeltaDDC).
|
| ColGroupEmpty |
|
| ColGroupFactory |
Factory class for constructing ColGroups.
|
| ColGroupIO |
IO for ColGroups, it enables read and write ColGroups
|
| ColGroupLinearFunctional |
|
| ColGroupOLE |
Class to encapsulate information about a column group that is encoded with simple lists of offsets for each set of
distinct values.
|
| ColGroupRLE |
A group of columns compressed with a single run-length encoded bitmap.
|
| ColGroupSDC |
Column group that sparsely encodes the dictionary values.
|
| ColGroupSDCFOR |
ColGroup for Patched Frame Of Reference.
|
| ColGroupSDCSingle |
Column group that sparsely encodes the dictionary values.
|
| ColGroupSDCSingleZeros |
Column group that sparsely encodes the dictionary values.
|
| ColGroupSDCZeros |
Column group that sparsely encodes the dictionary values.
|
| ColGroupSizes |
|
| ColGroupUncompressed |
Column group type for columns that are stored as dense arrays of doubles.
|
| ColGroupUncompressedArray |
Special sideways compressed column group not supposed to be used outside of the compressed transform encode.
|
| ColGroupUtils |
|
| ColGroupUtils.P |
|
| ColIndexes |
|
| ColIndexFactory |
|
| ColIndexStructure |
|
| ColIndexStructure.IndexProperties |
|
| CollectionUtils |
|
| ColumnEncoder |
Base class for all transform encoders providing both a row and block interface for decoding frames to matrices.
|
| ColumnEncoder.EncoderType |
|
| ColumnEncoderBagOfWords |
|
| ColumnEncoderBin |
|
| ColumnEncoderBin.BinMethod |
|
| ColumnEncoderComposite |
Simple composite encoder that applies a list of encoders in specified order.
|
| ColumnEncoderDummycode |
|
| ColumnEncoderFeatureHash |
Class used for feature hashing transformation of frames.
|
| ColumnEncoderFeatureHash.FeatureHashSparseApplyTask |
|
| ColumnEncoderPassThrough |
|
| ColumnEncoderPassThrough.PassThroughSparseApplyTask |
|
| ColumnEncoderRecode |
|
| ColumnEncoderUDF |
|
| ColumnEncoderWordEmbedding |
|
| ColumnMetadata |
|
| CombinedIndex |
|
| ComEstCompressed |
|
| ComEstCompressedSample |
|
| ComEstExact |
Exact compressed size estimator (examines entire dataset).
|
| ComEstFactory |
|
| ComEstSample |
Estimate compression size based on subsample of data.
|
| CommonThreadPool |
This common thread pool provides an abstraction to obtain a shared thread pool.
|
| CommonThreadPool.SameThreadExecutorService |
|
| CompilerConfig |
Basic wrapper for all compiler configurations that are configured
dynamically on a per script invocation basis.
|
| CompilerConfig.ConfigType |
|
| CompressedEncode |
|
| CompressedFrameBlockFactory |
|
| CompressedMatrixBlock |
|
| CompressedMatrixBlockFactory |
Factory pattern to compress a Matrix Block into a CompressedMatrixBlock.
|
| CompressedSizeInfo |
A helper reusable object for maintaining information about estimated compression
|
| CompressedSizeInfoColGroup |
Information collected about a specific ColGroup's compression size.
|
| CompressedWriteBlock |
Write block for serializing either a instance of MatrixBlock or CompressedMatrixBlock, To allow spark to read in
either or.
|
| Compression |
|
| Compression.CompressConfig |
|
| CompressionCPInstruction |
|
| CompressionDecoderEndStatisticsHandler |
|
| CompressionDecoderStartStatisticsHandler |
|
| CompressionEncoderEndStatisticsHandler |
|
| CompressionEncoderStartStatisticsHandler |
|
| CompressionPlanFactory |
|
| CompressionScheme |
A Class that contains a full compression scheme that can be applied to MatrixBlocks.
|
| CompressionSettings |
Compression Settings class, used as a bundle of parameters inside the Compression framework.
|
| CompressionSettingsBuilder |
Builder pattern for Compression Settings.
|
| CompressionSPInstruction |
|
| CompressionSPInstruction.CompressionFunction |
|
| CompressionSPInstruction.CompressionWorkloadFunction |
|
| CompressionSPInstruction.SizeFunction |
|
| CompressionStatistics |
Compression Statistics contain the main information gathered from the compression, such as sizes of the original
matrix, vs the compressed representation at different stages of the compression.
|
| CompressRDDClean |
|
| CompressUnwrap |
|
| CompressWrap |
|
| ComputationCostEstimator |
|
| ComputationCPInstruction |
|
| ComputationFEDInstruction |
|
| ComputationSPInstruction |
|
| ComputeBinaryBlockNnzFunction |
|
| ComputeCost |
Class with methods estimating compute costs of operations.
|
| ConditionalPredicate |
|
| ConfigurableAPI |
This interface defines the programmatic access to dml configuration properties
(as defined in SystemDS-config.xml) to ensure API consistency across all APIs.
|
| ConfigurationManager |
Singleton for accessing the parsed and merged system configuration.
|
| Connection |
Interaction with SystemDS using the JMLC (Java Machine Learning Connector) API is initiated with
a Connection object. |
| Constants |
|
| Constants |
|
| ConstEncoding |
Const encoding for cases where the entire group of columns is the same value
|
| ConstIdentifier |
|
| ConstScheme |
|
| Converter<K1 extends org.apache.hadoop.io.Writable,V1 extends org.apache.hadoop.io.Writable,K2 extends org.apache.hadoop.io.Writable,V2 extends org.apache.hadoop.io.Writable> |
|
| ConvertFrameBlockToIJVLines |
|
| ConvertMatrixBlockToIJVLines |
|
| ConvertStringToLongTextPair |
|
| CoordinatorConnectionModel |
|
| CoordinatorController |
|
| CoordinatorModel |
|
| CoordinatorService |
|
| CopyBinaryCellFunction |
|
| CopyFrameBlockFunction |
General purpose copy function for binary block rdds.
|
| CopyFrameBlockPairFunction |
General purpose copy function for binary block rdds.
|
| CopyMatrixBlockFunction |
General purpose copy function for binary block rdds.
|
| CopyMatrixBlockPairFunction |
General purpose copy function for binary block rdds.
|
| CopyTensorBlockFunction |
General purpose copy function for binary block rdds.
|
| CopyTensorBlockPairFunction |
General purpose copy function for binary block rdds.
|
| CopyTextInputFunction |
|
| CorrMatrixBlock |
|
| CostEstimationException |
Exception thrown when the cost estimation gets in
a state that should not raise runtime a exception.
|
| CostEstimationWrapper |
|
| CostEstimationWrapper.CostType |
|
| CostEstimator |
|
| CostEstimator |
Class for estimating the execution time of a program.
|
| CostEstimator |
Base class for all potential cost estimators
TODO account for shared read-only matrices when computing aggregated stats
|
| CostEstimator.DataFormat |
|
| CostEstimator.ExcludeType |
|
| CostEstimator.TestMeasure |
|
| CostEstimatorBuilder |
|
| CostEstimatorFactory |
Factory class for the construction of cost estimators for compression
|
| CostEstimatorFactory.CostType |
|
| CostEstimatorHops |
|
| CostEstimatorRuntime |
Cost estimator for runtime programs.
|
| CostEstimatorStaticRuntime |
|
| CountDistinctFunctionSketch |
|
| CountDistinctOperator |
|
| CountDistinctOperatorTypes |
|
| CountDistinctSketch |
|
| COV |
GENERAL NOTE:
* 05/28/2014: We decided to do handle weights consistently to SPSS in an operation-specific manner,
i.e., we (1) round instead of casting where required (e.g.
|
| CoVariance |
Lop to compute covariance between two 1D matrices
|
| CovarianceCPInstruction |
|
| CovarianceFEDInstruction |
|
| CovarianceSPInstruction |
|
| COVOperator |
|
| CPCostUtils |
|
| CPInstruction |
|
| CPInstruction.CPType |
|
| CPInstructionParser |
|
| CPlanCSERewriter |
|
| CPlanMemoTable |
|
| CPlanMemoTable.MemoTableEntry |
|
| CPlanMemoTable.MemoTableEntrySet |
|
| CPlanOpRewriter |
This cplan rewriter is meant to be the central place for any cplan
enhancements before code generation.
|
| CpmmSPInstruction |
Cpmm: cross-product matrix multiplication operation (distributed matrix multiply
by join over common dimension and subsequent aggregation of partial results).
|
| CPOperand |
|
| CreateSparseBlockFunction |
General purpose copy function for binary block values.
|
| CSRPointer |
Compressed Sparse Row (CSR) format for CUDA
Generalized matrix multiply is implemented for CSR format in the cuSparse library among other operations
Since we assume that the matrix is stored with zero-based indexing (i.e.
|
| CSVReBlock |
Lop to convert CSV data into SystemDS data format
|
| CSVReblockSPInstruction |
|
| Ctable |
Lop to perform ternary operation.
|
| CTable |
|
| Ctable.OperationTypes |
|
| CtableCPInstruction |
|
| CtableFEDInstruction |
|
| CTableMap |
Ctable map is an abstraction for the hashmap used for ctable's hash group-by
because this structure is passed through various interfaces.
|
| CtableSPInstruction |
|
| CtableSPInstruction.MapJoinSignature1 |
|
| CtableSPInstruction.MapJoinSignature2 |
|
| CtableSPInstruction.MapJoinSignature3 |
|
| CudaMemoryAllocator |
|
| CudaSupportFunctions |
DESIGN DOCUMENTATION FOR SUPPORTING LOWER PRECISION:
1.
|
| CumulativeAggregateSPInstruction |
|
| CumulativeOffsetBinary |
|
| CumulativeOffsetFEDInstruction |
|
| CumulativeOffsetSPInstruction |
|
| CumulativePartialAggregate |
|
| CustomErrorListener |
|
| CustomErrorListener.ParseIssueType |
Parse issues can be syntax errors, validation errors, and validation
warnings.
|
| CustomProperties |
|
| Dag<N extends Lop> |
Class to maintain a DAG of lops and compile it into
runtime instructions, incl piggybacking into jobs.
|
| Data |
Lop to represent data objects.
|
| Data |
|
| DataAugmentation |
|
| DataCharacteristics |
|
| DataConverter |
This class provides methods to read and write matrix blocks from to HDFS using different data formats.
|
| DataExpression |
|
| DataGen |
Defines a LOP that generates data.
|
| DataGenCPInstruction |
|
| DataGenOp |
A DataGenOp can be rand (or matrix constructor), sequence, and sample -
these operators have different parameters and use a map of parameter type to hop position.
|
| DataIdentifier |
|
| DataObjectModel |
|
| DataOp |
A DataOp can be either a persistent read/write or transient read/write - writes will always have at least one input,
but all types can have parameters (e.g., for csv literals of delimiter, header, etc).
|
| DataPartitioner |
This is the base class for all data partitioner.
|
| DataPartitionerLocal |
Partitions a given matrix into row or column partitions with a two pass-approach.
|
| DataPartitionerRemoteSpark |
MR job class for submitting parfor remote partitioning MR jobs.
|
| DataPartitionerRemoteSparkMapper |
NOTE: for the moment we only support binary block here
TODO extend impl for binarycell and textcell
Interface of Writable output in order to support both PairWritableBlock and PairWritableCell.
|
| DataPartitionerRemoteSparkReducer |
|
| DataPartitionerSparkAggregator |
|
| DataPartitionerSparkMapper |
|
| DataPartitionFederatedScheme |
|
| DataPartitionFederatedScheme.BalanceMetrics |
|
| DataPartitionFederatedScheme.Result |
|
| DataPartitionLocalScheme |
|
| DataPartitionSparkScheme |
|
| DatasetObject |
|
| DataTensorBlock |
|
| DblArray |
Helper class used for bitmap extraction.
|
| DblArrayCountHashMap |
|
| DblArrayIntListHashMap |
collect an array list of the indexes with the same hashes.
|
| DblArrayIntListHashMap.DArrayIListEntry |
|
| DCLocalScheme |
Disjoint_Contiguous data partitioner:
for each worker, use a right indexing
operation X[beg:end,] to obtain contiguous,
non-overlapping partitions of rows.
|
| DCSparkScheme |
Spark Disjoint_Contiguous data partitioner:
|
| DDCArray<T> |
A dense dictionary version of an column array
|
| DDCScheme |
|
| DDCSchemeMC |
|
| DDCSchemeSC |
|
| Decoder |
Base class for all transform decoders providing both a row and block
interface for decoding matrices to frames.
|
| DecoderBin |
Simple atomic decoder for binned columns.
|
| DecoderComposite |
Simple composite decoder that applies a list of decoders
in specified order.
|
| DecoderDummycode |
Simple atomic decoder for dummycoded columns.
|
| DecoderFactory |
|
| DecoderFactory.DecoderType |
|
| DecoderPassThrough |
Simple atomic decoder for passing through numeric columns to the output.
|
| DecoderRecode |
Simple atomic decoder for recoded columns.
|
| DeCompression |
|
| DeCompression.CompressConfig |
|
| DeCompressionCPInstruction |
|
| DeCompressionSPInstruction |
|
| DeCompressionSPInstruction.DeCompressionFunction |
|
| DeltaDictionary |
This dictionary class is a specialization for the DeltaDDCColgroup.
|
| DenseBlock |
This DenseBlock is an abstraction for different dense, row-major
matrix formats.
|
| DenseBlock.Type |
|
| DenseBlockBool |
|
| DenseBlockDRB |
|
| DenseBlockFactory |
|
| DenseBlockFP32 |
|
| DenseBlockFP64 |
|
| DenseBlockFP64DEDUP |
|
| DenseBlockInt32 |
|
| DenseBlockInt64 |
|
| DenseBlockLBool |
|
| DenseBlockLDRB |
Dense Large Row Blocks have multiple 1D arrays (blocks), which contain complete rows.
|
| DenseBlockLFP32 |
|
| DenseBlockLFP64 |
|
| DenseBlockLFP64DEDUP |
|
| DenseBlockLInt32 |
|
| DenseBlockLInt64 |
|
| DenseBlockLString |
|
| DenseBlockString |
|
| DenseEncoding |
An Encoding that contains a value on each row of the input.
|
| DependencyTask<E> |
|
| DependencyThreadPool |
|
| DependencyWrapperTask<E> |
|
| DerbyRepository |
|
| DiagIndex |
|
| Dictionary |
This dictionary class aims to encapsulate the storage and operations over unique floating point values of a column
group.
|
| DictionaryFactory |
|
| DictionaryFactory.Type |
|
| DictLibMatrixMult |
Utility interface for dictionary matrix multiplication
|
| DictWritable |
|
| DictWritable.K |
|
| DistinctCostEstimator |
A Cost based estimator that based the compression and co-coding cost on the number of distinct elements in the column
groups.
|
| Divide |
|
| DmlBaseListener |
This class provides an empty implementation of DmlListener,
which can be extended to create a listener which only needs to handle a subset
of the available methods. |
| DMLCompressionException |
|
| DMLCompressionStatistics |
|
| DMLConfig |
|
| DMLException |
Exception occurring in the DML framework.
|
| DmlLexer |
|
| DmlListener |
This interface defines a complete listener for a parse tree produced by
DmlParser. |
| DMLOptions |
Set of DMLOptions that can be set through the command line
and MLContext
The values have been initialized with the default values
Despite there being a DML and PyDML, this class is named DMLOptions
to keep it consistent with DMLOptions and DMLOptions |
| DmlParser |
|
| DmlParser.AccumulatorAssignmentStatementContext |
|
| DmlParser.AddSubExpressionContext |
|
| DmlParser.AssignmentStatementContext |
|
| DmlParser.AtomicExpressionContext |
|
| DmlParser.BooleanAndExpressionContext |
|
| DmlParser.BooleanNotExpressionContext |
|
| DmlParser.BooleanOrExpressionContext |
|
| DmlParser.BuiltinFunctionExpressionContext |
|
| DmlParser.CommandlineParamExpressionContext |
|
| DmlParser.CommandlinePositionExpressionContext |
|
| DmlParser.ConstDoubleIdExpressionContext |
|
| DmlParser.ConstFalseExpressionContext |
|
| DmlParser.ConstIntIdExpressionContext |
|
| DmlParser.ConstStringIdExpressionContext |
|
| DmlParser.ConstTrueExpressionContext |
|
| DmlParser.DataIdentifierContext |
|
| DmlParser.DataIdExpressionContext |
|
| DmlParser.DataTypeContext |
|
| DmlParser.ExpressionContext |
|
| DmlParser.ExternalFunctionDefExpressionContext |
|
| DmlParser.ForStatementContext |
|
| DmlParser.FunctionCallAssignmentStatementContext |
|
| DmlParser.FunctionCallMultiAssignmentStatementContext |
|
| DmlParser.FunctionStatementContext |
|
| DmlParser.IfdefAssignmentStatementContext |
|
| DmlParser.IfStatementContext |
|
| DmlParser.ImportStatementContext |
|
| DmlParser.IndexedExpressionContext |
|
| DmlParser.InternalFunctionDefExpressionContext |
|
| DmlParser.IterablePredicateColonExpressionContext |
|
| DmlParser.IterablePredicateContext |
|
| DmlParser.IterablePredicateSeqExpressionContext |
|
| DmlParser.MatrixDataTypeCheckContext |
|
| DmlParser.MatrixMulExpressionContext |
|
| DmlParser.Ml_typeContext |
|
| DmlParser.ModIntDivExpressionContext |
|
| DmlParser.MultDivExpressionContext |
|
| DmlParser.MultiIdExpressionContext |
|
| DmlParser.ParameterizedExpressionContext |
|
| DmlParser.ParForStatementContext |
|
| DmlParser.PathStatementContext |
|
| DmlParser.PowerExpressionContext |
|
| DmlParser.ProgramrootContext |
|
| DmlParser.RelationalExpressionContext |
|
| DmlParser.SimpleDataIdentifierExpressionContext |
|
| DmlParser.StatementContext |
|
| DmlParser.StrictParameterizedExpressionContext |
|
| DmlParser.StrictParameterizedKeyValueStringContext |
|
| DmlParser.TypedArgAssignContext |
|
| DmlParser.TypedArgNoAssignContext |
|
| DmlParser.UnaryExpressionContext |
|
| DmlParser.ValueTypeContext |
|
| DmlParser.WhileStatementContext |
|
| DMLParserWrapper |
This is the main entry point for the Antlr4 parser.
|
| DmlPreprocessor |
Minimal pre-processing of user function definitions which take precedence over built-in
functions in cases where names conflict.
|
| DMLProgram |
|
| DMLRuntimeException |
This exception should be thrown to flag runtime errors -- DML equivalent to java.lang.RuntimeException.
|
| DMLScript |
|
| DMLScriptException |
This exception should be thrown to flag DML Script errors, this exception is reserved for the stop instruction and
script related errors that an end-user should be able to address
|
| DmlSyntacticValidator |
|
| DmlSyntacticValidator.ConvertedDMLSyntax |
Information about built in functions converted to a common format between
PyDML and DML for the runtime.
|
| DMLTranslator |
|
| DMVUtils |
|
| DMVUtils.LEVEL_ENUM |
|
| DnnCPInstruction |
|
| DnnGPUInstruction |
|
| DnnOp |
|
| DnnParameters |
This class is container that stores parameters required for executing following operations:
conv2d, conv2d_backward_data, conv2d_backward_filter, maxpooling, maxpooling_backward, lstm, lstm_backward
|
| DnnSPInstruction |
|
| DnnTransform |
|
| DnnUtils |
|
| DocumentRepresentation |
|
| DoubleArray |
|
| DoubleBufferingOutputStream |
|
| DoubleCountHashMap |
|
| DoubleIdentifier |
|
| DoubleIntListHashMap |
This class provides a memory-efficient replacement for HashMap<Double,IntArrayList> for restricted use cases.
|
| DoubleIntListHashMap.DIListEntry |
|
| DoubleObject |
|
| DoubleParser |
A fast double parser inspired from https://github.com/wrandelshofer/FastDoubleParser
|
| DoubleParser.UInt128 |
|
| DoublePrecisionCudaSupportFunctions |
|
| DRLocalScheme |
Data partitioner Disjoint_Random:
for each worker, use a permutation multiply P[beg:end,] %*% X,
where P is constructed for example with P=table(seq(1,nrow(X)),sample(nrow(X), nrow(X))),
i.e., sampling without replacement to ensure disjointness.
|
| DRRLocalScheme |
Disjoint_Round_Robin data partitioner:
for each worker, use a permutation multiply
or simpler a removeEmpty such as removeEmpty
(target=X, margin=rows, select=(seq(1,nrow(X))%%k)==id)
|
| DRRSparkScheme |
Spark Disjoint_Round_Robin data partitioner:
|
| DRSparkScheme |
Spark data partitioner Disjoint_Random:
For the current row block, find all the shifted place for each row (WorkerID => (row block ID, matrix)
|
| EMAUtils |
|
| EmptyEncoding |
Empty encoding for cases where the entire group of columns is zero
|
| EmptyScheme |
|
| Encoder |
|
| EncoderFactory |
|
| EncoderMVImpute |
|
| EncoderMVImpute.MVMethod |
|
| EncoderOmit |
|
| EncodingFactory |
|
| Encrypted |
This class abstracts over an encrypted data.
|
| EnumerationUtils |
|
| EnumerationUtils.ConfigurationPoint |
Simple data structure to hold cluster configurations
|
| EnumerationUtils.InstanceSearchSpace |
Data structure representing a projected search space for
VM instances as node's memory mapped to further maps with
the node's numbers of cores for the given memory
mapped to a list of unique object of type CloudInstance
which have this corresponding characteristics (memory and cores).
|
| EnumerationUtils.SolutionPoint |
Data structure to hold all data related to cost estimation
|
| Enumerator |
|
| Enumerator.Builder |
|
| Enumerator.EnumerationStrategy |
|
| Enumerator.OptimizationStrategy |
|
| EnvironmentHelper |
This class is useful in setting environment variable for loading MKL library (done by Native Helper)
|
| Equals |
|
| EstimationFactors |
Compressed Size Estimation factors.
|
| EstimationUtils |
|
| EstimatorBasicAvg |
Basic average case estimator for matrix sparsity:
sp = 1 - Math.pow(1-sp1*sp2, k)
|
| EstimatorBasicWorst |
Basic average case estimator for matrix sparsity:
sp = Math.min(1, sp1 * k) * Math.min(1, sp2 * k).
|
| EstimatorBitsetMM |
This estimator implements a naive but rather common approach of boolean matrix
multiplies which allows to infer the exact non-zero structure and thus is
also useful for sparse result preallocation.
|
| EstimatorBitsetMM.BitsetMatrix |
|
| EstimatorBitsetMM.BitsetMatrix1 |
This class represents a boolean matrix and provides key operations.
|
| EstimatorBitsetMM.BitsetMatrix2 |
|
| EstimatorDensityMap |
This estimator implements an approach called density maps, as introduced in
David Kernert, Frank Köhler, Wolfgang Lehner: SpMacho - Optimizing Sparse
Linear Algebra Expressions with Probabilistic Density Estimation.
|
| EstimatorDensityMap.DensityMap |
|
| EstimatorLayeredGraph |
This estimator implements an approach based on a so-called layered graph,
introduced in
Edith Cohen.
|
| EstimatorLayeredGraph.LayeredGraph |
|
| EstimatorMatrixHistogram |
This estimator implements a remarkably simple yet effective
approach for incorporating structural properties into sparsity
estimation.
|
| EstimatorMatrixHistogram.MatrixHistogram |
|
| EstimatorSample |
This estimator implements an approach based on row/column sampling
Yongyang Yu, MingJie Tang, Walid G.
|
| EstimatorSampleRa |
This estimator implements an approach based on row/column sampling
Rasmus Resen Amossen, Andrea Campagna, Rasmus Pagh:
Better Size Estimation for Sparse Matrix Products.
|
| EvalNaryCPInstruction |
Eval built-in function instruction
Note: it supports only single matrix[double] output
|
| EventModel |
|
| EventStageModel |
|
| EvictCPInstruction |
|
| ExecutionConfig |
Java Wrapper to specify CUDA execution configuration for launching custom kernels
|
| ExecutionContext |
|
| ExecutionContextFactory |
|
| ExecutionContextMap |
|
| Explain |
|
| Explain.ExplainCounts |
|
| Explain.ExplainType |
|
| Expression |
|
| Expression.BinaryOp |
Binary operators.
|
| Expression.BooleanOp |
Boolean operators.
|
| Expression.DataOp |
Data operators.
|
| Expression.FunctCallOp |
Function call operators.
|
| Expression.RelationalOp |
Relational operators.
|
| ExpressionInfo |
This class exists solely to prevent compiler warnings.
|
| ExpressionList |
|
| ExtractBlockForBinaryReblock |
|
| ExtractGroup |
|
| ExtractGroup.ExtractGroupBroadcast |
|
| ExtractGroup.ExtractGroupJoin |
|
| ExtractGroupNWeights |
|
| FastBufferedDataInputStream |
|
| FastBufferedDataOutputStream |
This buffered output stream is essentially a merged version of
BufferedOutputStream and DataOutputStream, wrt SystemDS requirements.
|
| FastStringTokenizer |
This string tokenizer is essentially a simplified StringTokenizer.
|
| Federated |
|
| FederatedCompilationTimer |
|
| FederatedCompilationTimer.TimeEntry |
|
| FederatedCompressionStatistics |
|
| FederatedData |
|
| FederatedData.FederatedRequestEncoder |
|
| FederatedDataPartitioner |
|
| FederatedLocalData |
|
| FederatedLookupTable |
Lookup table mapping from a FedUniqueCoordID (funCID) to an
ExecutionContextMap (ECM) so that every coordinator can address federated
variables with its own local sequential variable IDs.
|
| FederatedMemoTable |
A Memoization Table for managing federated plans (FedPlan) based on combinations of Hops and fedOutTypes.
|
| FederatedMemoTable.FedPlan |
Represents a single federated execution plan with its associated costs and dependencies.
|
| FederatedMemoTable.FedPlanVariants |
Represents a collection of federated execution plan variants for a specific Hop and FederatedOutput.
|
| FederatedMemoTable.HopCommon |
Represents common properties and costs associated with a Hop.
|
| FederatedMemoTablePrinter |
|
| FederatedMonitoringServer |
|
| FederatedMonitoringServerHandler |
|
| FederatedPlanCostEnumerator |
Enumerates and evaluates all possible federated execution plans for a given Hop DAG.
|
| FederatedPlanCostEstimator |
Cost estimator for federated execution plans.
|
| FederatedPlannerFedAll |
Baseline federated planner that compiles all hops
that support federated execution on federated inputs to
forced federated operations.
|
| FederatedPlannerFedHeuristic |
|
| FederatedPlannerUtils |
Utility class for federated planners.
|
| FederatedPSControlThread |
|
| FederatedRange |
|
| FederatedReadCache |
|
| FederatedReadCache.ReadCacheEntry |
Class representing an entry of the federated read cache.
|
| FederatedRequest |
|
| FederatedRequest.RequestType |
|
| FederatedResponse |
|
| FederatedResponse.ResponseType |
|
| FederatedSSLUtil |
|
| FederatedStatistics |
|
| FederatedStatistics.FedStatsCollectFunction |
|
| FederatedStatistics.FedStatsCollection |
|
| FederatedUDF |
|
| FederatedWorker |
|
| FederatedWorker.FederatedResponseEncoder |
|
| FederatedWorkerHandler |
Note: federated worker handler created for every command; and concurrent parfor threads at coordinator need separate
execution contexts at the federated sites too
|
| FederatedWorkerHandlerException |
Exception to throw when an exception occurs in FederatedWorkerHandler during handling of FederatedRequest.
|
| FederatedWorkloadAnalyzer |
|
| FederationMap |
|
| FederationUtils |
|
| FEDInstruction |
|
| FEDInstruction.FederatedOutput |
|
| FEDInstruction.FEDType |
|
| FEDInstructionParser |
|
| FEDInstructionUtils |
|
| FileFormatProperties |
|
| FileFormatPropertiesCOG |
|
| FileFormatPropertiesCSV |
|
| FileFormatPropertiesHDF5 |
|
| FileFormatPropertiesLIBSVM |
|
| FileFormatPropertiesMM |
|
| FileFormatPropertiesMM.MMField |
|
| FileFormatPropertiesMM.MMFormat |
|
| FileFormatPropertiesMM.MMSymmetry |
|
| FilterDiagMatrixBlocksFunction |
|
| FilterNonEmptyBlocksFunction |
|
| FilterNonEmptyBlocksFunction2 |
|
| FloatArray |
|
| FormatIdentifyer |
|
| ForProgramBlock |
|
| ForStatement |
|
| ForStatementBlock |
|
| Frame |
Frame encapsulates a SystemDS frame.
|
| FrameAppendCPInstruction |
|
| FrameAppendMSPInstruction |
|
| FrameAppendRSPInstruction |
|
| FrameBlock |
|
| FrameBlock.FrameMapFunction |
|
| FrameCodeGen |
|
| FrameCompressionSettings |
|
| FrameCompressionSettingsBuilder |
|
| FrameFormat |
FrameFormat represents the different frame formats supported by the MLContext
API.
|
| FrameFromMatrixBlock |
|
| FrameGenerateReader |
|
| FrameGenerateReaderParallel |
|
| FrameIndexingCPInstruction |
|
| FrameIndexingSPInstruction |
This class implements the frame indexing functionality inside Spark.
|
| FrameLibAppend |
|
| FrameLibApplySchema |
|
| FrameLibCompress |
|
| FrameLibDetectSchema |
|
| FrameLibRemoveEmpty |
|
| FrameMetadata |
Frame metadata, such as the number of rows, the number of columns, the number
of non-zero values, the number of rows per block, and the number of columns
per block in the frame.
|
| FrameObject |
|
| FrameRDDAggregateUtils |
|
| FrameRDDConverterUtils |
|
| FrameRDDConverterUtils.LongFrameToLongWritableFrameFunction |
|
| FrameRDDConverterUtils.LongWritableFrameToLongFrameFunction |
|
| FrameRDDConverterUtils.LongWritableTextToLongTextFunction |
|
| FrameRDDConverterUtils.LongWritableToSerFunction |
|
| FrameReader |
Base class for all format-specific frame readers.
|
| FrameReaderBinaryBlock |
Single-threaded frame binary block reader.
|
| FrameReaderBinaryBlockParallel |
Multi-threaded frame binary block reader.
|
| FrameReaderFactory |
|
| FrameReaderJSONL |
|
| FrameReaderJSONLParallel |
|
| FrameReaderProto |
|
| FrameReaderTextCell |
Single-threaded frame textcell reader.
|
| FrameReaderTextCellParallel |
Multi-threaded frame textcell reader.
|
| FrameReaderTextCSV |
Single-threaded frame text csv reader.
|
| FrameReaderTextCSVParallel |
Multi-threaded frame text csv reader.
|
| FrameReblockBuffer |
|
| FrameSchema |
The frame schema, stored as a list of ValueType values.
|
| FrameUtil |
|
| FrameWriter |
Base class for all format-specific frame writers.
|
| FrameWriterBinaryBlock |
Single-threaded frame binary block writer.
|
| FrameWriterBinaryBlockParallel |
Multi-threaded frame binary block writer.
|
| FrameWriterCompressed |
|
| FrameWriterFactory |
|
| FrameWriterJSONL |
|
| FrameWriterJSONLParallel |
|
| FrameWriterProto |
|
| FrameWriterTextCell |
Single-threaded frame text cell writer.
|
| FrameWriterTextCellParallel |
Multi-threaded frame text cell writer.
|
| FrameWriterTextCSV |
Single-threaded frame text csv writer.
|
| FrameWriterTextCSVParallel |
Single-threaded frame text csv writer.
|
| FTypes |
|
| FTypes.AlignType |
|
| FTypes.FederatedPlanner |
|
| FTypes.FPartitioning |
|
| FTypes.FReplication |
|
| FTypes.FType |
|
| FunctionCallCP |
|
| FunctionCallCPInstruction |
|
| FunctionCallGraph |
|
| FunctionCallIdentifier |
|
| FunctionCallSizeInfo |
Auxiliary data structure to hold function call summaries in terms
of information about number of function calls, consistent dimensions,
consistent sparsity, and dimension-preserving functions.
|
| FunctionDictionary<T extends Types.FunctionBlock> |
Dictionary of all functions of a namespace, represented as a simple
key-value map of function names and function statement blocks.
|
| FunctionObject |
|
| FunctionOp |
This FunctionOp represents the call to a DML-bodied or external function.
|
| FunctionOp.FunctionType |
|
| FunctionProgramBlock |
|
| FunctionStatement |
|
| FunctionStatementBlock |
|
| GenerateReader |
|
| GenerateReader.GenerateReaderFrame |
|
| GenerateReader.GenerateReaderMatrix |
|
| GPUContext |
Represents a context per GPU accessible through the same JVM.
|
| GPUContextPool |
|
| GPUInstruction |
|
| GPUInstruction.GPUINSTRUCTION_TYPE |
|
| GPUInstructionParser |
|
| GPULazyCudaFreeMemoryManager |
|
| GPUMatrixMemoryManager |
|
| GPUMemoryAllocator |
|
| GPUMemoryEviction |
|
| GPUMemoryManager |
- All cudaFree and cudaMalloc in SystemDS should go through this class to avoid OOM or incorrect results.
|
| GPUMemoryManager.EvictionPolicyBasedComparator |
Class that governs the eviction policy
|
| GPUObject |
Handle to a matrix block on the GPU
|
| GPUStatistics |
Measures performance numbers when GPU mode is enabled
Printed as part of Statistics. |
| GreaterThan |
|
| GreaterThanEquals |
|
| GridBasedEnumerator |
|
| GroupedAggregate |
Lop to perform grouped aggregates
|
| GroupedAggregateM |
Lop to perform mr map-side grouped aggregates
(restriction: sum, w/o weights, ngroups), groups broadcasted
|
| GroupedAggregateM.CacheType |
|
| H5 |
|
| H5BTree |
|
| H5BufferBuilder |
|
| H5Constants |
|
| H5ContiguousDataset |
|
| H5DataLayoutMessage |
|
| H5DataSpaceMessage |
|
| H5DataTypeMessage |
|
| H5DoubleDataType |
|
| H5FillValueMessage |
|
| H5GroupSymbolTableNode |
|
| H5LocalHeap |
|
| H5Message |
|
| H5NilMessage |
|
| H5ObjectHeader |
|
| H5ObjectModificationTimeMessage |
|
| H5RootObject |
|
| H5RuntimeException |
|
| H5Superblock |
|
| H5SymbolTableEntry |
|
| H5SymbolTableMessage |
|
| Hash |
A class containing different hashing functions.
|
| Hash.HashType |
Available Hashing techniques
|
| HashIntegerArray |
|
| HashLongArray |
|
| HashMapLongInt |
|
| HashMapToInt<K> |
|
| HassAndStokes |
|
| HassAndStokes.MethodOfMomentsFunction |
|
| HDFSTool |
|
| HeavyHitterModel |
|
| HEParamServer |
This class implements Homomorphic Encryption (HE) for LocalParamServer.
|
| Hop |
Hop is a High level operator, that is the first intermediate representation compiled from the definitions supplied in
DML.
|
| HopDagValidator |
This class allows to check hop dags for validity, e.g., parent-child linking.
|
| HopRewriteRule |
Base class for all hop rewrites in order to enable generic
application of all rules.
|
| HopRewriteUtils |
|
| HopsException |
Exception occurring in the HOP level.
|
| HybridCostEstimator |
|
| ICLAScheme |
Interface for a scheme instance.
|
| IColIndex |
Class to contain column indexes for the compression column groups.
|
| IColIndex.ColIndexType |
|
| IColIndex.SliceResult |
A Class for slice results containing indexes for the slicing of dictionaries, and the resulting column index
|
| IContainADictionary |
|
| IContainDefaultTuple |
|
| IController |
|
| IDagLinearizer |
An interface for the linearization algorithms that order the DAG nodes into a
sequence of instructions to execute.
|
| IDagLinearizerFactory |
|
| IDagLinearizerFactory.DagLinearizer |
|
| Identifier |
|
| IdentityDictionary |
A specialized dictionary that exploits the fact that the contained dictionary is an Identity Matrix.
|
| IdentityDictionarySlice |
|
| IDHandler |
Functionalities for extracting numeric IDs from Hadoop taskIDs and other
things related to modification of IDs.
|
| IDictionary |
|
| IDictionary.DictType |
|
| IDSequence |
ID sequence for generating unique long identifiers with start 0 and increment 1.
|
| IEncode |
This interface covers an intermediate encoding for the samples to improve the efficiency of the joining of sample
column groups.
|
| IFDTag |
Represents a single tag in the IFD of a TIFF file
|
| IFDTagDictionary |
Enum for mapping IFD tag values to their corresponding tag names
|
| IfElse |
|
| IfProgramBlock |
|
| IFrameOfReferenceGroup |
Interface for frame of reference groups.
|
| IfStatement |
|
| IfStatementBlock |
|
| IHashArray |
|
| IIterate |
Class to iterate through the columns of a IColIndex.
|
| IJV |
Helper class for external key/value exchange.
|
| IMapToDataGroup |
|
| ImportStatement |
|
| IndexedIdentifier |
|
| IndexedMatrixValue |
|
| IndexedTensorBlock |
|
| IndexFunction |
|
| IndexingCPInstruction |
|
| IndexingFEDInstruction |
|
| IndexingOp |
|
| IndexingSPInstruction |
This class implements the matrix indexing functionality inside Spark.
|
| IndexRange |
|
| InfrastructureAnalyzer |
Central place for analyzing and obtaining static infrastructure properties
such as memory and number of logical processors.
|
| InitFEDInstruction |
|
| InputOutputInfo |
|
| InputStreamInputFormat |
Custom input format and record reader to redirect common implementation of csv read
over record readers (which are required for the parallel readers) to an input stream.
|
| InsertionSorterFactory |
|
| InsertionSorterFactory.SORT_TYPE |
|
| InstallDependencyForIntegrationTests |
This class will be used to setup dependency on Eclipse environment as well as on Jenkins server
|
| Instruction |
|
| Instruction.IType |
|
| InstructionParser |
|
| InstructionType |
|
| InstructionTypeCounter |
|
| InstructionUtils |
|
| IntArrayList |
|
| IntegerArray |
|
| IntegerDivide |
|
| InterestBasedEnumerator |
|
| InterestingPoint |
Interesting decision point with regard to materialization of intermediates.
|
| InterestingPoint.DecisionType |
|
| InterProceduralAnalysis |
This Inter Procedural Analysis (IPA) serves two major purposes:
1) Inter-Procedure Analysis: propagate statistics from calling program into
functions and back into main program.
|
| IntIdentifier |
|
| IntObject |
|
| IOCostUtils |
|
| IOCostUtils.IOMetrics |
|
| IOUtilFunctions |
|
| IOUtilFunctions.CountRowsTask |
|
| IPAPass |
Base class for all IPA passes.
|
| IPAPassApplyStaticAndDynamicHopRewrites |
This rewrite applies static hop dag and statement block
rewrites such as constant folding and branch removal
in order to simplify statistic propagation.
|
| IPAPassCompressionWorkloadAnalysis |
This rewrite obtains workload summaries for all hops candidates amenable for compression as a basis for
workload-aware compression planning.
|
| IPAPassEliminateDeadCode |
This rewrite eliminates unnecessary sub-DAGs that produce
transient write outputs which are never consumed subsequently.
|
| IPAPassFlagFunctionsRecompileOnce |
This rewrite marks functions with loops as recompile once
in order to reduce recompilation overhead.
|
| IPAPassFlagLoopsRecompileOnce |
This rewrite marks loops in the main program as recompile once
in order to reduce recompilation overhead.
|
| IPAPassFlagNonDeterminism |
|
| IPAPassForwardFunctionCalls |
This rewrite forwards a function call to a function with a simple
function call that only consumes function parameters and literals
into the original call location.
|
| IPAPassInlineFunctions |
This rewrite inlines single statement block functions, which have fewer
operations than an internal threshold.
|
| IPAPassPropagateReplaceLiterals |
This rewrite propagates and replaces literals into functions
in order to enable subsequent rewrites such as branch removal.
|
| IPAPassRemoveConstantBinaryOps |
This rewrite identifies binary operations with constant matrices
such as X * ones, where ones might be created as a vector of ones
before a loop.
|
| IPAPassRemoveUnnecessaryCheckpoints |
This rewrite identifies and removes unnecessary checkpoints, i.e.,
persisting of Spark RDDs into a given storage level.
|
| IPAPassRemoveUnusedFunctions |
This rewrite identifies and removes unused functions in order
to reduce compilation overhead and other overheads such as
parfor worker creation, where we construct function copies.
|
| IPAPassReplaceEvalFunctionCalls |
This rewrite applies static hop dag and statement block
rewrites such as constant folding and branch removal
in order to simplify statistic propagation.
|
| IPAPassRewriteFederatedPlan |
This rewrite generates a federated execution plan by estimating and setting costs and the FederatedOutput values of
all relevant hops in the DML program.
|
| IPlanEncode |
Interface for constructing an compression plan that is able to compress sequences of matrix blocks with the same
scheme
|
| IRepository |
|
| IsBlockInList |
|
| IsBlockInRange |
|
| IsFrameBlockInRange |
|
| ISliceOffset |
|
| IterablePredicate |
|
| IteratorFactory |
Factory pattern for construction of rowIterators of the FrameBlock.
|
| JCudaKernels |
|
| JMLCUtils |
Utility class containing static methods for working with JMLC.
|
| JSONHelper |
|
| KahanFunction |
Runtime function type to perform the summation of values using
the Kahan summation algorithm.
|
| KahanObject |
|
| KahanPlus |
|
| KahanPlusSq |
Runtime function to perform the summation of squared values using
the Kahan summation algorithm.
|
| KeepDataOnWorkerFederatedScheme |
Keep Data on Worker Federated scheme
When the parameter server runs in federated mode it cannot pull in the data which is already on the workers.
|
| KMVSketch |
KMV synopsis(for k minimum values) Distinct-Value Estimation
Kevin S.
|
| LanguageException |
Exception occurring at the Language level.
|
| LanguageException.LanguageErrorCodes |
|
| LazyIterableIterator<T> |
This class is a generic base class for lazy, single pass iterator classes
in order to simplify the implementation of lazy iterators for mapPartitions
use cases.
|
| LazyWriteBuffer |
|
| LazyWriteBuffer.RPolicy |
|
| LeftIndex |
|
| LeftIndex.LixCacheType |
|
| LeftIndexingOp |
|
| LeftIndexingOp.LeftIndexingMethod |
|
| LeftScalarOperator |
Scalar operator for scalar-matrix operations with scalar
on the left-hand-side.
|
| LegacyEncoder |
Base class for all transform encoders providing both a row and block interface for decoding frames to matrices.
|
| LessThan |
|
| LessThanEquals |
|
| LibCommonsMath |
Library for matrix operations that need invocation of
Apache Commons Math library.
|
| LibMatrixAgg |
MB:
Library for matrix aggregations including ak+, uak+ for all
combinations of dense and sparse representations, and corrections.
|
| LibMatrixAggUnarySpecialization |
This class handel the generic case of aggregate Unary operations, it is only used in cases not optimized in
LibMatrixAgg.
|
| LibMatrixAppend |
|
| LibMatrixBincell |
Library for binary cellwise operations (incl arithmetic, relational, etc).
|
| LibMatrixBincell.BinaryAccessType |
|
| LibMatrixCountDistinct |
This class contains various methods for counting the number of distinct values inside a MatrixBlock
|
| LibMatrixCUDA |
All CUDA kernels and library calls are redirected through this class
|
| LibMatrixCuDNN |
This class contains method that invoke CuDNN operations.
|
| LibMatrixCuDNNConvolutionAlgorithm |
This class is a wrapper that contain necessary data structures to invoke
a cudnn convolution* functions (such as cudnnConvolutionForward, etc)
It implements autocloseable to simplify the LibMatrixCuDNN code and also avoids potential memory leaks.
|
| LibMatrixCuDNNInputRowFetcher |
Performs a slice operation: out = in[(n+1):(n+1), 1:numColumns]
|
| LibMatrixCuDNNPoolingDescriptors |
This class is a wrapper that contain necessary data structures to invoke
a cudnn convolution* functions (such as cudnnConvolutionForward, etc)
It implements autocloseable to simplify the LibMatrixCuDNN code and also avoids potential memory leaks.
|
| LibMatrixCuDNNRnnAlgorithm |
|
| LibMatrixCuMatMult |
|
| LibMatrixDatagen |
|
| LibMatrixDenseToSparse |
|
| LibMatrixDNN |
|
| LibMatrixDNN.PoolingType |
|
| LibMatrixDNNConv2d |
This class contains the set of operators used for performing conv2d
|
| LibMatrixDNNHelper |
|
| LibMatrixDNNIm2Col |
This class contains the different implementation of im2col operation
|
| LibMatrixDNNLSTM |
|
| LibMatrixDNNPooling |
This class contains the set of operators used for performing pooling
|
| LibMatrixDNNRelu |
This class contains the different implementation of rotate180 operation
|
| LibMatrixDNNRelu.ReluBackward |
Performs the operation: (X gt 0) * dout
|
| LibMatrixDNNRotate180 |
This class contains the different implementation of rotate180 operation
|
| LibMatrixDNNRotate180.Rotate180Worker |
|
| LibMatrixEquals |
Equals library for MatrixBlocks:
|
| LibMatrixFourier |
|
| LibMatrixMult |
MB: Library for matrix multiplications including MM, MV, VV for all
combinations of dense, sparse, ultrasparse representations and special
operations such as transpose-self matrix multiplication.
|
| LibMatrixNative |
|
| LibMatrixOuterAgg |
ACS:
Purpose of this library is to make some of the unary outer aggregate operator more efficient.
|
| LibMatrixReorg |
MB:
Library for selected matrix reorg operations including special cases
and all combinations of dense and sparse representations.
|
| LibMatrixReplace |
|
| LibMatrixSketch |
|
| LibMatrixSparseToDense |
|
| LibMatrixSTFT |
Liberary file containing methods to perform short time fourier transformations.
|
| LibMatrixTercell |
Library for ternary cellwise operations.
|
| LibSpoofPrimitives |
This library contains all vector primitives that are used in
generated source code for fused operators.
|
| LIBSVMReblockSPInstruction |
|
| LibTensorAgg |
|
| LibTensorBincell |
|
| LibTensorReorg |
|
| Lineage |
|
| LineageCache |
|
| LineageCacheConfig |
|
| LineageCacheConfig.LineageCachePolicy |
|
| LineageCacheConfig.ReuseCacheType |
|
| LineageCacheEntry |
|
| LineageCacheEviction |
|
| LineageCacheStatistics |
This singleton provides basic lineage caching statistics in CP.
|
| LineageCodegenItem |
|
| LineageDebugger |
|
| LineageDedupBlock |
|
| LineageDedupUtils |
|
| LineageEstimator |
|
| LineageEstimatorStatistics |
|
| LineageGPUCacheEviction |
|
| LineageItem |
|
| LineageItem.LineageItemType |
|
| LineageItemUtils |
|
| LineageMap |
|
| LineageObject |
|
| LineageParser |
|
| LineageRecomputeUtils |
|
| LineageRewriteReuse |
|
| LineageSparkCacheEviction |
|
| LineageTokenizer |
|
| LineageTraceable |
|
| LinearizerBreadthFirst |
|
| LinearizerCostBased |
|
| LinearizerDepthFirst |
|
| LinearizerMaxParallelism |
|
| LinearizerMinIntermediates |
Sort lops to execute them in an order that
minimizes the memory requirements of intermediates
|
| LinearizerPipelineAware |
|
| LinearizerResourceAwareFast |
|
| LinearizerResourceAwareOptimal |
|
| LinearRegression |
|
| ListAppendRemoveCPInstruction |
|
| ListIdentifier |
|
| ListIndexingCPInstruction |
|
| ListObject |
|
| ListReader |
|
| ListWriter |
|
| LiteralOp |
|
| LiteralReplacement |
|
| LiveVariableAnalysis |
|
| Local |
|
| LocalCPInstruction |
A simple instruction that take whatever input it is and return it as a local matrix.
|
| LocalDataPartitioner |
|
| LocalFileUtils |
|
| LocalParamServer |
|
| LocalParWorker |
Instances of this class can be used to execute tasks in parallel.
|
| LocalPSWorker |
|
| LocalTaskQueue<T> |
This class provides a way of dynamic task distribution to multiple workers
in local multi-threaded environments.
|
| LocalVariableMap |
Replaces HashMap⟨String, Data⟩ as the table of
variable names and references.
|
| LongArray |
|
| LongestCommonSubsequence |
|
| LongLongDoubleHashMap |
This native long long - double hashmap is specifically designed for
ctable operations which only require addvalue - extract semantics.
|
| LongLongDoubleHashMap.ADoubleEntry |
|
| LongLongDoubleHashMap.EntryType |
|
| Lop |
Base class for all Lops.
|
| Lop.SimpleInstType |
Lop types
|
| Lop.Type |
|
| Lop.VisitStatus |
|
| LopComparator<N extends Lop> |
Comparator class used in sorting the LopDAG in topological order.
|
| LopProperties |
|
| LopRewriter |
|
| LopRewriteRule |
|
| LopsException |
|
| MapInputSignature |
|
| MapJoinSignature |
|
| MapmmChainSPInstruction |
|
| MapmmSPInstruction |
|
| MapMult |
|
| MapMult.CacheType |
|
| MapMultChain |
|
| MapMultChain.ChainType |
|
| MapperService |
|
| MappingProperties |
|
| MappingProperties.DataProperties |
|
| MappingProperties.RecordProperties |
|
| MappingProperties.RepresentationProperties |
|
| MappingTrie |
|
| MappingTrieNode |
|
| MappingTrieNode.Type |
|
| MapToBit |
A Bit map containing the values of the map inside a long array.
|
| MapToByte |
|
| MapToChar |
|
| MapToCharPByte |
|
| MapToFactory |
Interface for the factory design pattern for construction all AMapToData.
|
| MapToFactory.MAP_TYPE |
The different supported types of mappings.
|
| MapToInt |
|
| MapToUByte |
|
| MapToZero |
|
| MarkForLineageReuse |
|
| MaterializeSort |
|
| MatMultCP |
|
| Matrix |
Matrix encapsulates a SystemDS matrix.
|
| MatrixAppendCPInstruction |
|
| MatrixAppendGPUInstruction |
Implements the cbind and rbind functions for matrices
|
| MatrixAppendMSPInstruction |
|
| MatrixAppendRSPInstruction |
|
| MatrixBlock |
|
| MatrixBlockDataInput |
Any data input that is intended to support fast deserialization / read
of entire blocks should implement this interface.
|
| MatrixBlockDataOutput |
Any data output that is intended to support fast serialization / write
of entire blocks should implement this interface.
|
| MatrixBlockDictionary |
|
| MatrixBlockFromFrame |
|
| MatrixBuiltinGPUInstruction |
|
| MatrixBuiltinNaryCPInstruction |
|
| MatrixCell |
|
| MatrixCharacteristics |
|
| MatrixCodeGen |
|
| MatrixFormat |
MatrixFormat represents the different matrix formats supported by the
MLContext API.
|
| MatrixGenerateReader |
|
| MatrixGenerateReaderParallel |
|
| MatrixIndexes |
This represent the indexes to the blocks of the matrix.
|
| MatrixIndexingCPFileInstruction |
This instruction is used if a single partition is too large to fit in memory.
|
| MatrixIndexingCPInstruction |
|
| MatrixIndexingGPUInstruction |
|
| MatrixIndexingSPInstruction |
This class implements the matrix indexing functionality inside CP.
|
| MatrixLineagePair |
Class to represent the relation between a MatrixObject and the respective
LineageItem.
|
| MatrixMatrixArithmeticGPUInstruction |
|
| MatrixMatrixAxpyGPUInstruction |
|
| MatrixMatrixBinaryOpFunction |
|
| MatrixMatrixBuiltinGPUInstruction |
|
| MatrixMatrixRelationalBinaryGPUInstruction |
|
| MatrixMetadata |
Matrix metadata, such as the number of rows, the number of columns, the
number of non-zero values, the number of rows per block, and the number of
columns per block in the matrix.
|
| MatrixObject |
Represents a matrix in control program.
|
| MatrixObject.UpdateType |
|
| MatrixObjectFuture |
|
| MatrixReader |
Base class for all format-specific matrix readers.
|
| MatrixReaderFactory |
|
| MatrixReshapeGPUInstruction |
|
| MatrixReshapeSPInstruction |
|
| MatrixScalarUnaryFunction |
|
| MatrixSketch |
|
| MatrixSketchFactory |
|
| MatrixValue |
|
| MatrixValue.CellIndex |
|
| MatrixVectorBinaryOpPartitionFunction |
|
| MatrixWriter |
Base class for all format-specific matrix writers.
|
| MatrixWriterFactory |
|
| Max |
|
| Mean |
|
| Memorizer |
|
| MemorizerV2 |
|
| MemoryCostEstimator |
|
| MemoryEstimates |
Memory Estimates is a helper class containing static classes that estimate the memory requirements of different types
of objects in java.
|
| MemoryMonitor |
This memory monitor periodically calls garbage collection
in order to obtain the actual memory consumption during
runtime.
|
| MemoTable |
Memoization Table (hop id, worst-case matrix characteristics).
|
| MergeSort |
|
| Metadata |
Abstract metadata class for MLContext API.
|
| MetaData |
Class to store metadata associated with a file (e.g., a matrix) on disk.
|
| MetaDataAll |
|
| MetaDataFormat |
|
| MetaDataUtils |
|
| Min |
|
| Minus |
|
| Minus1Multiply |
|
| MinusMultiply |
|
| MinusNz |
|
| MLContext |
The MLContext API offers programmatic access to SystemDS on Spark from
languages such as Scala, Java, and Python.
|
| MLContext.ExecutionType |
The different types of execution environments supported by SystemDS.
|
| MLContext.ExplainLevel |
The different explain levels supported by SystemDS.
|
| MLContextConversionUtil |
Utility class containing methods to perform data conversions.
|
| MLContextException |
Uncaught exception representing SystemDS exceptions that occur through the
MLContext API.
|
| MLContextProxy |
The purpose of this proxy is to shield systemds internals from direct access to MLContext
which would try to load spark libraries and hence fail if these are not available.
|
| MLContextUtil |
Utility class containing methods for working with the MLContext API.
|
| MLResults |
MLResults handles the results returned from executing a Script using the
MLContext API.
|
| MMChainCPInstruction |
|
| MMChainFEDInstruction |
|
| MMCJ |
Lop to perform cross product operation
|
| MMCJ.MMCJType |
|
| MMFEDInstruction |
|
| MMNode |
Helper class to represent matrix multiply operators in a DAG
along with references to its abstract data handles.
|
| MMRJ |
Lop to perform cross product operation
|
| MMTSJ |
Lop to perform transpose-identity operation (t(X)%*%X or X%*%t(X)),
used to represent CP and MR instruction but in case of MR there is
an additional Aggregate at the reducers.
|
| MMTSJ.MMTSJType |
|
| MMTSJCPInstruction |
|
| MMTSJGPUInstruction |
|
| MMZip |
Lop to perform zip matrix multiplication
|
| Modulus |
Integer modulus, where we adhere to the defined R semantics:
("%% indicates x mod y and %/% indicates integer division.
|
| MultiAssignmentStatement |
|
| MultiColBitmap |
Uncompressed representation of one or more columns in bitmap format.
|
| MultiColumnEncoder |
|
| Multiply |
|
| Multiply2 |
|
| MultiReturnBuiltinCPInstruction |
|
| MultiReturnComplexMatrixBuiltinCPInstruction |
|
| MultiReturnParameterizedBuiltinCPInstruction |
|
| MultiReturnParameterizedBuiltinFEDInstruction |
|
| MultiReturnParameterizedBuiltinFEDInstruction.CreateFrameEncoder |
|
| MultiReturnParameterizedBuiltinFEDInstruction.ExecuteFrameEncoder |
|
| MultiReturnParameterizedBuiltinSPInstruction |
|
| MultiReturnParameterizedBuiltinSPInstruction.TransformEncodeBuild2Function |
|
| MultiReturnParameterizedBuiltinSPInstruction.TransformEncodeBuildFunction |
This function pre-aggregates distinct values of recoded columns per partition (part of distributed recode map
construction, used for recoding, binning and dummy coding).
|
| MultiReturnParameterizedBuiltinSPInstruction.TransformEncodeGroup2Function |
|
| MultiReturnParameterizedBuiltinSPInstruction.TransformEncodeGroupFunction |
This function assigns codes to globally distinct values of recoded columns and writes the resulting column map in
textcell (IJV) format to the output.
|
| MultiThreadedHop |
Optional hop interface, to be implemented by multi-threaded hops.
|
| MultiThreadedOperator |
|
| NaivePlanEncode |
Naive implementation of encoding based on a plan.
|
| Nary |
Lop to perform an operation on a variable number of operands.
|
| NaryOp |
The NaryOp Hop allows for a variable number of operands.
|
| NativeHEHelper |
|
| NativeHelper |
This class helps in loading native library.
|
| NativeHelper.NativeBlasState |
|
| NativeStatistics |
|
| NetworkTrafficCounter |
|
| NGramBuilder<T,U> |
|
| NGramBuilder.NGramEntry<T,U> |
|
| NormalPRNGenerator |
Class that can generate a stream of random numbers from standard
normal distribution N(0,1).
|
| Not |
|
| NotEquals |
|
| ObjectRowIterator |
|
| OffsetByte |
|
| OffsetByteNZ |
|
| OffsetByteUNZ |
|
| OffsetChar |
|
| OffsetColumnIndex |
|
| OffsetEmpty |
|
| OffsetFactory |
|
| OffsetFactory.OFF_TYPE |
The specific underlying types of offsets.
|
| OffsetFactory.OFF_TYPE_SPECIALIZATIONS |
Specialized types of underlying offsets.
|
| OffsetSingle |
|
| OffsetTwo |
|
| Op |
|
| Opcodes |
|
| OperationsOnMatrixValues |
|
| Operator |
|
| OperatorOrderingUtils |
|
| OpMetadata |
|
| OpNormal |
|
| OpSided |
|
| OptimizationWrapper |
Wrapper to ParFOR cost estimation and optimizer.
|
| Optimizer |
Generic optimizer super class that defines the interface of all implemented optimizers.
|
| Optimizer.CostModelType |
|
| OptimizerConstrained |
Rule-Based ParFor Optimizer (time: O(n)):
Applied rule-based rewrites:
- see base class.
|
| OptimizerHeuristic |
Heuristic ParFor Optimizer: This optimizer extends the rule-based
optimizer by a time-based cost estimate for execution type decisions.
|
| OptimizerRuleBased |
Rule-Based ParFor Optimizer (time: O(n)):
Applied rule-based rewrites
- 1) rewrite set data partitioner (incl.
|
| OptimizerUtils |
|
| OptimizerUtils.MemoryManager |
Memory managers (static partitioned, unified)
|
| OptimizerUtils.OptimizationLevel |
Optimization Types for Compilation
O0 STATIC - Decisions for scheduling operations on CP/MR are based on
predefined set of rules, which check if the dimensions are below a
fixed/static threshold (OLD Method of choosing between CP and MR).
|
| OptionalArray<T> |
|
| OptNode |
Internal representation of a plan alternative for program blocks and instructions
in order to enable efficient and simple recursive enumeration and plan changes.
|
| OptNode.ExecType |
|
| OptNode.NodeType |
|
| OptNode.ParamType |
|
| OptTree |
Represents a complete plan of a top-level parfor.
|
| OptTreeConverter |
Converter for creating an internal plan representation for a given runtime program
and to modify/create the runtime program according to the optimized plan.
|
| OptTreePlanMapping |
|
| Or |
|
| ORLocalScheme |
Data partitioner Overlap_Reshuffle:
for each worker, use a new permutation multiply P %*% X,
where P is constructed for example with P=table(seq(1,nrow(X),sample(nrow(X), nrow(X))))
|
| ORSparkScheme |
Spark data partitioner Overlap_Reshuffle:
|
| OuterVectorBinaryOpFunction |
|
| OutputParameters |
class to maintain output parameters for a lop.
|
| OutputStatement |
|
| PageCache |
|
| Pair<K,V> |
|
| PairWritableBlock |
Custom writable for a pair of matrix indexes and matrix block
as required for binaryblock in remote data partitioning.
|
| PairWritableCell |
Custom writable for a pair of matrix indexes and matrix cell
as required for binarycell in remote data partitioning.
|
| ParameterBuilder |
Class to help setting variables in a script.
|
| ParameterExpression |
|
| ParameterizedBuiltin |
Defines a LOP for functions.
|
| ParameterizedBuiltin |
Function object for builtin function that takes a list of name=value parameters.
|
| ParameterizedBuiltin.ParameterizedBuiltinCode |
|
| ParameterizedBuiltin.ProbabilityDistributionCode |
|
| ParameterizedBuiltinCPInstruction |
|
| ParameterizedBuiltinFEDInstruction |
|
| ParameterizedBuiltinFEDInstruction.DecodeMatrix |
|
| ParameterizedBuiltinFunctionExpression |
|
| ParameterizedBuiltinOp |
Defines the HOP for calling an internal function (with custom parameters) from a DML script.
|
| ParameterizedBuiltinSPInstruction |
|
| ParameterizedBuiltinSPInstruction.CreateMatrixCell |
|
| ParameterizedBuiltinSPInstruction.RDDContainsFunction |
|
| ParameterizedBuiltinSPInstruction.RDDContainsVectFunction |
|
| ParameterizedBuiltinSPInstruction.RDDFrameReplaceFunction |
|
| ParameterizedBuiltinSPInstruction.RDDMapGroupedAggFunction |
|
| ParameterizedBuiltinSPInstruction.RDDMapGroupedAggFunction2 |
Similar to RDDMapGroupedAggFunction but single output block.
|
| ParameterizedBuiltinSPInstruction.RDDRemoveEmptyFunction |
|
| ParameterizedBuiltinSPInstruction.RDDRemoveEmptyFunctionInMem |
|
| ParameterizedBuiltinSPInstruction.RDDReplaceFunction |
|
| ParameterizedBuiltinSPInstruction.RDDRExpandFunction |
|
| ParameterizedBuiltinSPInstruction.RDDTokenizeFunction |
|
| ParameterizedBuiltinSPInstruction.RDDTransformApplyFunction |
|
| ParameterizedBuiltinSPInstruction.RDDTransformApplyFunction2 |
|
| ParameterizedBuiltinSPInstruction.RDDTransformApplyOffsetFunction |
|
| ParameterizedBuiltinSPInstruction.RDDTransformDecodeExpandFunction |
|
| ParameterizedBuiltinSPInstruction.RDDTransformDecodeFunction |
|
| ParamservBuiltinCPInstruction |
|
| ParamServer |
|
| ParamServStatistics |
|
| ParamservUtils |
|
| ParForBody |
Wrapper for exchanging parfor body data structures.
|
| ParForProgramBlock |
The ParForProgramBlock has the same execution semantics as a ForProgamBlock but executes
the independent iterations in parallel.
|
| ParForProgramBlock.PartitionFormat |
Convenience class to package PDataPartitionFormat and its parameters.
|
| ParForProgramBlock.PDataPartitioner |
|
| ParForProgramBlock.PDataPartitionFormat |
|
| ParForProgramBlock.PExecMode |
|
| ParForProgramBlock.POptMode |
|
| ParForProgramBlock.PResultMerge |
|
| ParForProgramBlock.PTaskPartitioner |
|
| ParForStatement |
This ParForStatement is essentially identical to a ForStatement, except an extended
toString method for printing the 'parfor' keyword.
|
| ParForStatementBlock |
This ParForStatementBlock is essentially identical to a ForStatementBlock, except an extended validate
for checking/setting optional parfor parameters and running the loop dependency analysis.
|
| ParForStatementBlock.ResultVar |
|
| ParForStatistics |
|
| ParseException |
This exception is thrown when parse issues are encountered.
|
| ParseInfo |
|
| ParserFactory |
|
| ParserWrapper |
Base class for all dml parsers in order to make the various compilation chains
independent of the used parser.
|
| PartialAggregate |
Lop to perform a partial aggregation.
|
| PartitionedBlock<T extends CacheBlock> |
This class is for partitioned matrix/frame blocks, to be used as broadcasts.
|
| PartitionedBroadcast<T extends CacheBlock> |
This class is a wrapper around an array of broadcasts of partitioned matrix/frame blocks,
which is required due to 2GB limitations of Spark's broadcast handling.
|
| ParWorker |
Super class for master/worker pattern implementations.
|
| PathStatement |
|
| PerformGroupByAggInCombiner |
|
| PerformGroupByAggInReducer |
|
| PickByCount |
|
| PickByCount.OperationTypes |
|
| PlaceHolderDict |
|
| PlaintextMatrix |
This class abstracts over an encrypted matrix of ciphertexts.
|
| PlanAnalyzer |
Utility functions to extract structural information from the memo table,
including connected components (aka partitions) of partial fusion plans,
materialization points of partitions, and root nodes of partitions.
|
| PlanPartition |
|
| PlanSelection |
|
| PlanSelection.VisitMarkCost |
|
| PlanSelectionFuseAll |
This plan selection heuristic aims for maximal fusion, which
potentially leads to overlapping fused operators and thus,
redundant computation but with a minimal number of materialized
intermediate results.
|
| PlanSelectionFuseCostBased |
This cost-based plan selection algorithm chooses fused operators
based on the DAG structure and resulting overall costs.
|
| PlanSelectionFuseCostBasedV2 |
This cost-based plan selection algorithm chooses fused operators
based on the DAG structure and resulting overall costs.
|
| PlanSelectionFuseNoRedundancy |
This plan selection heuristic aims for fusion without any redundant
computation, which, however, potentially leads to more materialized
intermediates than the fuse all heuristic.
|
| Plus |
|
| PlusMultiply |
|
| PMapmmSPInstruction |
This pmapmm matrix multiplication instruction is still experimental
not integrated in automatic operator selection yet.
|
| PMapMult |
|
| PMMJ |
|
| PMMJ.CacheType |
|
| PMMJCPInstruction |
|
| PmmSPInstruction |
|
| PoissonPRNGenerator |
Class that can generate a stream of random numbers from Poisson
distribution with specified mean.
|
| PorterStemmer |
Stemmer, implementing the Porter Stemming Algorithm
The Stemmer class transforms a word into its root form.
|
| Power |
|
| Power2 |
|
| PrefetchCPInstruction |
|
| PreparedScript |
Representation of a prepared (precompiled) DML/PyDML script.
|
| PrintStatement |
|
| PrintStatement.PRINTTYPE |
The PRINTTYPE options are: PRINT, PRINTF, and STOP.
|
| PRNGenerator |
|
| Program |
|
| ProgramBlock |
|
| ProgramConverter |
Program converter functionalities for
(1) creating deep copies of program blocks, instructions, function program blocks, and
(2) serializing and parsing of programs, program blocks, functions program blocks.
|
| ProgramRecompiler |
|
| ProgramRewriter |
This program rewriter applies a variety of rule-based rewrites
on all hop dags of the given program in one pass over the entire
program.
|
| ProgramRewriteStatus |
|
| ProjectInfo |
Obtains information that is stored in the manifest when the SystemDS jar is
built.
|
| PruneBasedEnumerator |
|
| PSRpcCall |
|
| PSRpcFactory |
|
| PSRpcHandler |
|
| PSRpcObject |
|
| PSRpcResponse |
|
| PSRpcResponse.Type |
|
| PSWorker |
|
| PublicKey |
|
| Py4jConverterUtils |
Utils for converting python data to java.
|
| PythonDMLScript |
|
| QDictionary |
This dictionary class aims to encapsulate the storage and operations over unique floating point values of a column
group.
|
| QuantilePickCPInstruction |
|
| QuantilePickFEDInstruction |
|
| QuantilePickFEDInstruction.CreateMatrixFromFrame |
|
| QuantilePickFEDInstruction.GetHistogram |
|
| QuantilePickFEDInstruction.GetValuesInRange |
|
| QuantilePickFEDInstruction.GetValuesInRanges |
|
| QuantilePickFEDInstruction.MinMax |
|
| QuantilePickSPInstruction |
|
| QuantileSortCPInstruction |
This class supports two variants of sort operation on a 1-dimensional input matrix.
|
| QuantileSortFEDInstruction |
|
| QuantileSortSPInstruction |
This class supports two variants of sort operation on a 1-dimensional input matrix.
|
| QuaternaryCPInstruction |
|
| QuaternaryFEDInstruction |
|
| QuaternaryOp |
Note: this hop should be called AggQuaternaryOp in consistency with AggUnaryOp and AggBinaryOp;
however, since there does not exist a real QuaternaryOp yet - we can leave it as is for now.
|
| QuaternaryOperator |
|
| QuaternarySPInstruction |
|
| QuaternaryWCeMMFEDInstruction |
|
| QuaternaryWDivMMFEDInstruction |
|
| QuaternaryWSigmoidFEDInstruction |
|
| QuaternaryWSLossFEDInstruction |
|
| QuaternaryWUMMFEDInstruction |
|
| RaggedArray<T> |
A Ragged array for a single column contains a smaller array, only containing the values of the top most part of the
This makes the allocation much better in cases where only the top n rows of a m row frame are used for the specific
column.
|
| RandNPair |
Class that generates a pair of random numbers from standard normal
distribution N(0,1).
|
| RandomMatrixGenerator |
|
| RandomMatrixGenerator.PDF |
Types of Probability density functions
|
| RandSPInstruction |
|
| RangeIndex |
A Range index that contain a lower and upper bound of the indexes that is symbolize.
|
| RawIndex |
|
| RDDAggregateUtils |
Collection of utility methods for aggregating binary block rdds.
|
| RDDConverterUtils |
|
| RDDConverterUtils.BinaryCellToBinaryBlockFunction |
|
| RDDConverterUtils.DataFrameExtractIDFunction |
|
| RDDConverterUtilsExt |
NOTE: These are experimental converter utils.
|
| RDDConverterUtilsExt.AddRowID |
|
| RDDConverterUtilsExt.RDDConverterTypes |
|
| RDDObject |
|
| RDDSortUtils |
|
| RDDSortUtils.IndexComparator |
|
| RDDSortUtils.IndexComparator2 |
|
| RDDStats |
|
| ReachabilityGraph |
|
| ReachabilityGraph.SubProblem |
|
| ReaderBinaryBlock |
|
| ReaderBinaryBlockParallel |
|
| ReaderCOG |
|
| ReaderCOGParallel |
|
| ReaderColumnSelection |
Base class for all column selection readers.
|
| ReaderColumnSelectionDenseMultiBlock |
|
| ReaderColumnSelectionDenseMultiBlockTransposed |
|
| ReaderColumnSelectionDenseSingleBlock |
|
| ReaderColumnSelectionDenseSingleBlockTransposed |
|
| ReaderColumnSelectionEmpty |
|
| ReaderColumnSelectionSparse |
Used to extract the values at certain indexes from each row in a sparse matrix
Keeps returning all-zeros arrays until reaching the last possible index.
|
| ReaderColumnSelectionSparseTransposed |
Used to extract the values at certain indexes from each row in a sparse matrix
Keeps returning all-zeros arrays until reaching the last possible index.
|
| ReaderCompressed |
|
| ReaderHDF5 |
|
| ReaderHDF5Parallel |
|
| ReaderMapping |
|
| ReaderMappingIndex |
|
| ReaderSparkCompressed |
|
| ReaderTextCell |
|
| ReaderTextCellParallel |
Parallel version of ReaderTextCell.java.
|
| ReaderTextCellParallel.CellBuffer |
Useful class for buffering unordered cells before locking target onces and
appending all buffered cells.
|
| ReaderTextCellParallel.CountNnzTask |
|
| ReaderTextCellParallel.ReadTask |
|
| ReaderTextCSV |
|
| ReaderTextCSVParallel |
Parallel version of ReaderTextCSV.java.
|
| ReaderTextLIBSVM |
|
| ReaderTextLIBSVMParallel |
|
| ReaderWriterFederated |
This class serves as the reader for federated objects.
|
| ReadProperties |
|
| ReBlock |
Lop to perform reblock operation
|
| ReblockBuffer |
|
| ReblockFEDInstruction |
|
| ReblockSPInstruction |
|
| ReblockTensorFunction |
|
| Recompiler |
Dynamic recompilation of hop dags to runtime instructions, which includes the
following substeps:
(1) deep copy hop dag, (2) refresh matrix characteristics, (3) apply
dynamic rewrites, (4) refresh memory estimates, (5) construct lops (incl
operator selection), and (6) generate runtime program (incl piggybacking).
|
| Recompiler.ResetType |
|
| RecompileStatistics |
|
| RecompileStatus |
|
| RecomputeNnzFunction |
|
| ReduceAll |
Reduce both columns and rows into a single cell
|
| ReduceCol |
Reduce the columns into fewer columns, normally 1
|
| ReduceDiag |
|
| ReduceRow |
Reduce all rows into a single row.
|
| RelationalBinaryGPUInstruction |
|
| RelationalExpression |
|
| RemoteDPParForSpark |
TODO heavy hitter maintenance
TODO data partitioning with binarycell
|
| RemoteDPParForSparkWorker |
|
| RemoteParForJobReturn |
Wrapper for job return of ParFor REMOTE for transferring statistics and result symbol table.
|
| RemoteParForSpark |
This class serves two purposes: (1) isolating Spark imports to enable running in
environments where no Spark libraries are available, and (2) to follow the same
structure as the parfor remote_mr job submission.
|
| RemoteParForSparkWorker |
|
| RemoteParForUtils |
Common functionalities for parfor workers in MR jobs.
|
| ReorgCPInstruction |
|
| ReorgFEDInstruction |
|
| ReorgFEDInstruction.DiagMatrix |
|
| ReorgFEDInstruction.Rdiag |
|
| ReorgFEDInstruction.SliceMatrix |
|
| ReorgGPUInstruction |
|
| ReorgMapFunction |
|
| ReorgOp |
Reorg (cell) operation: aij
Properties:
Symbol: ', rdiag, rshape, rsort
1 Operand (except sort and reshape take additional arguments)
Semantic: change indices (in mapper or reducer)
NOTE MB: reshape integrated here because (1) ParameterizedBuiltinOp requires name-value pairs for params
and (2) most importantly semantic of reshape is exactly a reorg op.
|
| ReorgOperator |
|
| ReorgSPInstruction |
|
| ReplicateBlockFunction |
|
| ReplicateTensorFunction |
|
| ReplicateToMaxFederatedScheme |
Replicate to Max Federated scheme
When the parameter server runs in federated mode it cannot pull in the data which is already on the workers.
|
| ReplicateVectorFunction |
|
| Request |
|
| RequestModel |
|
| ReshapeCPInstruction |
|
| ReshapeFEDInstruction |
|
| ResourceCompiler |
This class does full or partial program recompilation
based on given runtime program.
|
| ResourceOptimizer |
|
| Response |
|
| ResultMerge<T extends CacheableData<?>> |
|
| ResultMergeFrameLocalMemory |
|
| ResultMergeLocalAutomatic |
|
| ResultMergeLocalFile |
|
| ResultMergeLocalMemory |
Local in-memory realization of result merge.
|
| ResultMergeMatrix |
Due to independence of all iterations, any result has the following properties:
|
| ResultMergeRemoteSpark |
|
| ResultMergeRemoteSparkWCompare |
|
| ResultVariables |
A ResultVariables object holds the data returned by a call
to PreparedScript's executeScript
method, which executes a DML/PyDML script. |
| RevIndex |
|
| RewriteAddBroadcastLop |
|
| RewriteAddChkpointInLoop |
|
| RewriteAddChkpointLop |
|
| RewriteAddGPUEvictLop |
|
| RewriteAddPrefetchLop |
|
| RewriteAlgebraicSimplificationDynamic |
Rule: Algebraic Simplifications.
|
| RewriteAlgebraicSimplificationStatic |
Rule: Algebraic Simplifications.
|
| RewriteBlockSizeAndReblock |
Rule: BlockSizeAndReblock.
|
| RewriteCommonSubexpressionElimination |
Rule: CommonSubexpressionElimination.
|
| RewriteCompressedReblock |
Rule: Compressed Re block if config compressed.linalg is enabled, we inject compression directions after read of
matrices if number of rows is above 1000 and cols at least 1.
|
| RewriteConstantFolding |
Rule: Constant Folding.
|
| RewriteElementwiseMultChainOptimization |
Prerequisite: RewriteCommonSubexpressionElimination must run before this rule.
|
| RewriteFixIDs |
|
| RewriteForLoopVectorization |
Rule: Simplify program structure by pulling if or else statement body out
(removing the if statement block ifself) in order to allow intra-procedure
analysis to propagate exact statistics.
|
| RewriteGPUSpecificOps |
|
| RewriteHoistLoopInvariantOperations |
Rule: Simplify program structure by hoisting loop-invariant operations
out of while, for, or parfor loops.
|
| RewriteIndexingVectorization |
Rule: Indexing vectorization.
|
| RewriteInjectSparkLoopCheckpointing |
Rule: Insert checkpointing operations for caching purposes.
|
| RewriteInjectSparkPReadCheckpointing |
Rule: Inject checkpointing on reading in data in all cases where the operand is used in more than one operation.
|
| RewriteMarkLoopVariablesUpdateInPlace |
Rule: Mark loop variables that are only read/updated through cp left indexing
for update in-place.
|
| RewriteMatrixMultChainOptimization |
Rule: Determine the optimal order of execution for a chain of
matrix multiplications
Solution: Classic Dynamic Programming
Approach: Currently, the approach based only on matrix dimensions
Goal: To reduce the number of computations in the run-time
(map-reduce) layer
|
| RewriteMatrixMultChainOptimizationSparse |
Rule: Determine the optimal order of execution for a chain of
matrix multiplications
Solution: Classic Dynamic Programming
Approach: Currently, the approach based only on matrix dimensions
and sparsity estimates using the MNC sketch
Goal: To reduce the number of computations in the run-time
(map-reduce) layer
|
| RewriteMatrixMultChainOptimizationTranspose |
Rule: Determine the optimal order of execution for a chain of
matrix multiplications
Solution: Classic Dynamic Programming
Approach: Currently, the approach based only on matrix dimensions
Goal: To reduce the number of computations in the run-time
(map-reduce) layer
|
| RewriteMergeBlockSequence |
Rule: Simplify program structure by merging sequences of last-level
statement blocks in order to create optimization opportunities.
|
| RewriteNonScalarPrint |
|
| RewriteRemoveEmptyBasicBlocks |
Rule: Simplify program structure by removing empty last-level blocks,
which may originate from the original program or due to a sequence of
rewrites (e.g., checkpoint injection and subsequent IPA).
|
| RewriteRemoveEmptyForLoops |
Rule: Simplify program structure by removing empty for loops,
which may originate from the sequence of other rewrites like
dead-code-elimination.
|
| RewriteRemoveForLoopEmptySequence |
Rule: Simplify program structure by removing (par)for statements iterating over
an empty sequence, i.e., (par)for-loops without a single iteration.
|
| RewriteRemovePersistentReadWrite |
This rewrite is a custom rewrite for JMLC in order to replace all persistent reads
and writes with transient reads and writes from the symbol table.
|
| RewriteRemoveReadAfterWrite |
Rule: RemoveReadAfterWrite.
|
| RewriteRemoveTransformEncodeMeta |
Rule: If transformencode procudes a meta data frame which is never
used, flag transformencode to never allocate an serialize this frame.
|
| RewriteRemoveUnnecessaryBranches |
Rule: Simplify program structure by pulling if or else statement body out
(removing the if statement block ifself) in order to allow intra-procedure
analysis to propagate exact statistics.
|
| RewriteRemoveUnnecessaryCasts |
Rule: RemoveUnnecessaryCasts.
|
| RewriteSplitDagDataDependentOperators |
Rule: Split Hop DAG after specific data-dependent operators.
|
| RewriteSplitDagUnknownCSVRead |
Rule: Split Hop DAG after CSV reads with unknown size.
|
| RewriteUpdateGPUPlacements |
|
| RightIndex |
|
| RightScalarOperator |
Scalar operator for scalar-matrix operations with scalar
on the right-hand-side.
|
| RLEScheme |
|
| RmmSPInstruction |
|
| RollIndex |
This index function is NOT used for actual sorting but just as a reference
in ReorgOperator in order to identify sort operations.
|
| RowIndexStructure |
|
| RowIndexStructure.IndexProperties |
|
| RowIterator<T> |
|
| RowMatrixBlock |
|
| SampleEstimatorFactory |
|
| SampleEstimatorFactory.EstimationType |
|
| SampleFormatDataTypes |
Enum for mapping sample formats of TIFF image data to names
|
| SampleProperties |
|
| ScalarAppendCPInstruction |
|
| ScalarBuiltinNaryCPInstruction |
The ScalarBuiltinMultipleCPInstruction class is responsible for printf-style
Java-based string formatting.
|
| ScalarMatrixArithmeticGPUInstruction |
|
| ScalarMatrixBuiltinGPUInstruction |
|
| ScalarMatrixRelationalBinaryGPUInstruction |
|
| ScalarObject |
|
| ScalarObjectFactory |
|
| ScalarOperator |
Base class for all scalar operators.
|
| SchemeFactory |
|
| Script |
A Script object encapsulates a DML or PYDML script.
|
| ScriptExecutor |
ScriptExecutor executes a DML or PYDML Script object using SystemDS.
|
| ScriptExecutorUtils |
|
| ScriptFactory |
Factory for creating DML and PYDML Script objects from strings, files, URLs,
and input streams.
|
| SDCScheme |
|
| SDCSchemeMC |
|
| SDCSchemeSC |
|
| SEALClient |
|
| SEALServer |
|
| SerLongWritable |
Wrapper for LongWritable in order to make it serializable as required for
shuffle in spark instructions.
|
| SerText |
Wrapper for Text in order to make it serializable as required for
shuffle in spark instructions.
|
| SettingsChecker |
Settings Checker class that contains checks for the JVM setting of systemds when executed.
|
| ShadowBuffer |
|
| ShlosserEstimator |
|
| ShlosserJackknifeEstimator |
|
| ShlosserJackknifeEstimator.CriticalValue |
|
| ShuffleFederatedScheme |
Shuffle Federated scheme
When the parameter server runs in federated mode it cannot pull in the data which is already on the workers.
|
| SimpleOperator |
|
| SingleIndex |
|
| SinglePrecisionCudaSupportFunctions |
|
| SingletonLookupHashMap |
This class allows sharing of objects across the entire program.
|
| SmallestPriorityQueue |
Deceiving name, but is used to contain the k smallest values inserted.
|
| SmoothedJackknifeEstimator |
|
| SortIndex |
This index function is NOT used for actual sorting but just as a reference
in ReorgOperator in order to identify sort operations.
|
| SortKeys |
|
| SortKeys.OperationTypes |
|
| SortUtils |
Utilities for sorting, primarily used for SparseRows.
|
| SparkCostUtils |
|
| SparkDataPartitioner |
|
| SparkExecutionContext |
|
| SparkExecutionContext.MemoryManagerParRDDs |
|
| SparkExecutionContext.SparkClusterConfig |
Captures relevant spark cluster configuration properties, e.g., memory budgets and
degree of parallelism.
|
| SparkParamservUtils |
|
| SparkPSBody |
Wrapper class containing all needed for launching spark remote worker
|
| SparkPSProxy |
|
| SparkPSWorker |
|
| SparkStatistics |
|
| SparkUtils |
|
| SparseBlock |
This SparseBlock is an abstraction for different sparse matrix formats.
|
| SparseBlock.Type |
|
| SparseBlockCOO |
SparseBlock implementation that realizes a traditional 'coordinate matrix'
representation, where the entire sparse block is stored as triples in three arrays:
row indexes, column indexes, and values, where row indexes and colunm indexes are
sorted in order to allow binary search.
|
| SparseBlockCSC |
SparseBlock implementation that realizes a traditional 'compressed sparse column' representation, where the entire
sparse block is stored as three arrays: ptr of length clen+1 to store offsets per column, and indexes/values of
length nnz to store row indexes and values of non-zero entries.
|
| SparseBlockCSR |
SparseBlock implementation that realizes a traditional 'compressed sparse row'
representation, where the entire sparse block is stored as three arrays: ptr
of length rlen+1 to store offsets per row, and indexes/values of length nnz
to store column indexes and values of non-zero entries.
|
| SparseBlockDCSR |
|
| SparseBlockFactory |
|
| SparseBlockMCSC |
SparseBlock implementation that realizes a 'modified compressed sparse column' representation, where each compressed
column is stored as a separate SparseRow object which provides flexibility for unsorted column appends without the
need for global reshifting of values/indexes but it incurs additional memory overhead per column for object/array
headers per column which also slows down memory-bound operations due to higher memory bandwidth requirements.
|
| SparseBlockMCSR |
SparseBlock implementation that realizes a 'modified compressed sparse row'
representation, where each compressed row is stored as a separate SparseRow
object which provides flexibility for unsorted row appends without the need
for global reshifting of values/indexes but it incurs additional memory
overhead per row for object/array headers per row which also slows down
memory-bound operations due to higher memory bandwidth requirements.
|
| SparseEncoding |
A Encoding that contain a default value that is not encoded and every other value is encoded in the map.
|
| SparseRow |
Base class for sparse row implementations such as sparse
row vectors and sparse scalars (single value per row).
|
| SparseRowScalar |
|
| SparseRowVector |
A sparse row vector that is able to grow dynamically as values are appended to it.
|
| SparsityEstimator |
|
| SparsityEstimator.OpCode |
|
| SPInstruction |
|
| SPInstruction.SPType |
|
| SPInstructionParser |
|
| SpoofCellwise |
|
| SpoofCellwise.AggOp |
|
| SpoofCellwise.CellType |
|
| SpoofCompiler |
|
| SpoofCompiler.CompilerType |
|
| SpoofCompiler.GeneratorAPI |
|
| SpoofCompiler.IntegrationType |
|
| SpoofCompiler.PlanCachePolicy |
|
| SpoofCompiler.PlanSelector |
|
| SpoofCompiler.RegisterAlloc |
|
| SpoofCPInstruction |
|
| SpoofCUDACellwise |
|
| SpoofCUDAInstruction |
|
| SpoofCUDAInstruction.DoublePrecision |
|
| SpoofCUDAInstruction.SinglePrecision |
|
| SpoofCUDAOperator |
|
| SpoofCUDAOperator.PrecisionProxy |
|
| SpoofCUDARowwise |
|
| SpoofFEDInstruction |
|
| SpoofFused |
|
| SpoofFusedOp |
|
| SpoofFusedOp.SpoofOutputDimsType |
|
| SpoofMultiAggregate |
|
| SpoofOperator |
|
| SpoofOperator.SideInput |
|
| SpoofOperator.SideInputSparseCell |
|
| SpoofOperator.SideInputSparseRow |
|
| SpoofOuterProduct |
|
| SpoofOuterProduct.OutProdType |
|
| SpoofRowwise |
|
| SpoofRowwise.RowType |
|
| SpoofSPInstruction |
|
| SpoofSPInstruction.ReplicateRightFactorFunction |
|
| Sql |
|
| SqlCPInstruction |
|
| Statement |
|
| Statement.FederatedPSScheme |
|
| Statement.PSCheckpointing |
|
| Statement.PSFrequency |
|
| Statement.PSModeType |
|
| Statement.PSRuntimeBalancing |
|
| Statement.PSScheme |
|
| Statement.PSUpdateType |
|
| StatementBlock |
|
| StatementBlockRewriteRule |
Base class for all hop rewrites in order to enable generic
application of all rules.
|
| StatementInfo |
This class exists solely to prevent compiler warnings.
|
| Statistics |
This class captures all statistics.
|
| Statistics.LineageNGramExtension |
|
| Statistics.NGramStats |
|
| StatisticsController |
|
| StatisticsModel |
|
| StatisticsOptions |
|
| StatisticsService |
|
| StringArray |
|
| StringIdentifier |
|
| StringInitCPInstruction |
|
| StringObject |
|
| StringRowIterator |
|
| SubsampleToMinFederatedScheme |
Subsample to Min Federated scheme
When the parameter server runs in federated mode it cannot pull in the data which is already on the workers.
|
| SwapIndex |
|
| Task |
A task is a logical group of one or multiple iterations (each iteration is assigned to exactly one task).
|
| Task.TaskType |
|
| TaskPartitioner |
This is the base class for all task partitioner.
|
| TaskPartitionerFactoring |
This factoring task partitioner virtually iterates over the given FOR loop (from, to, incr),
creates iterations and group them to tasks.
|
| TaskPartitionerFactoringCmax |
Factoring with maximum constraint (e.g., if LIX matrix out-of-core and we need
to bound the maximum number of iterations per map task -> memory bounds)
|
| TaskPartitionerFactoringCmin |
Factoring with minimum constraint (e.g., if communication is expensive)
|
| TaskPartitionerFactory |
|
| TaskPartitionerFixedsize |
This naive task partitioner virtually iterates over the given FOR loop (from, to, incr),
creates iterations and group them to tasks according to the given task size.
|
| TaskPartitionerNaive |
This static task partitioner virtually iterates over the given FOR loop (from, to, incr),
creates iterations and group them to tasks according to a task size of numIterations/numWorkers.
|
| TaskPartitionerStatic |
This static task partitioner virtually iterates over the given FOR loop (from, to, incr),
creates iterations and group them to tasks according to a task size of numIterations/numWorkers.
|
| TemplateBase |
|
| TemplateBase.CloseType |
|
| TemplateBase.TemplateType |
|
| TemplateCell |
|
| TemplateCell.HopInputComparator |
Comparator to order input hops of the cell template.
|
| TemplateCodeGenBase |
|
| TemplateMultiAgg |
|
| TemplateOuterProduct |
|
| TemplateRow |
|
| TemplateRow.HopInputComparator |
Comparator to order input hops of the row aggregate template.
|
| TemplateUtil |
|
| TemplateUtil.SplitInfo |
|
| TemplateUtil.SplitOffsetInfos |
|
| TemplateUtils |
|
| TensorBlock |
A TensorBlock is the most top level representation of a tensor.
|
| TensorCharacteristics |
|
| TensorIndexes |
This represent the indexes to the blocks of the tensor.
|
| TensorObject |
|
| TensorReader |
|
| TensorReaderBinaryBlock |
|
| TensorReaderBinaryBlockParallel |
|
| TensorReaderFactory |
|
| TensorReaderTextCell |
|
| TensorReaderTextCellParallel |
|
| TensorTensorBinaryOpFunction |
|
| TensorTensorBinaryOpPartitionFunction |
|
| TensorWriter |
|
| TensorWriterBinaryBlock |
|
| TensorWriterBinaryBlockParallel |
|
| TensorWriterBinaryBlockParallel.WriteFileTask |
|
| TensorWriterFactory |
|
| TensorWriterTextCell |
|
| TensorWriterTextCellParallel |
|
| Ternary |
|
| Ternary |
|
| Ternary |
Lop to perform Sum of a matrix with another matrix multiplied by Scalar.
|
| TernaryAggregate |
|
| TernaryCPInstruction |
|
| TernaryFEDInstruction |
|
| TernaryFrameScalarCPInstruction |
|
| TernaryFrameScalarFEDInstruction |
|
| TernaryFrameScalarSPInstruction |
|
| TernaryOp |
Primary use cases for now, are
quantile (<n-1-matrix>, <n-1-matrix>, <literal>): quantile (A, w, 0.5)
quantile (<n-1-matrix>, <n-1-matrix>, <scalar>): quantile (A, w, s)
interquantile (<n-1-matrix>, <n-1-matrix>, <scalar>): interquantile (A, w, s)
Keep in mind, that we also have binaries for it w/o weights.
|
| TernaryOperator |
|
| TernarySPInstruction |
|
| TernaryValueFunction |
|
| TextTrie |
|
| TextTrieNode |
|
| TfMetaUtils |
|
| TfOffsetMap |
|
| TfUtils |
|
| TfUtils.TfMethod |
|
| TIFFDataTypes |
Enum for mapping data types of IFD tags in TIFF to readable names
|
| Timing |
Helper class to time the execution of elements
|
| Token |
|
| Token.SubToken |
|
| Tokenizer |
|
| TokenizerApplier |
|
| TokenizerApplierCount |
|
| TokenizerApplierHash |
|
| TokenizerApplierPosition |
|
| TokenizerBuilder |
|
| TokenizerBuilderNgram |
|
| TokenizerBuilderWhitespaceSplit |
|
| TokenizerFactory |
|
| TrafficModel |
|
| Transform |
|
| TransformStatistics |
|
| TriggerBroadcastTask |
|
| TriggerCheckpointTask |
|
| TriggerPrefetchTask |
|
| TripleIndexes |
|
| TripleIndexes.Comparator |
|
| Tsmm2SPInstruction |
|
| TsmmFEDInstruction |
|
| TsmmSPInstruction |
|
| TwoIndex |
|
| TwoRangesIndex |
|
| Types |
Common type information for the entire SystemDS.
|
| Types.AggOp |
Aggregation operations
|
| Types.BlockType |
Serialization block types (empty, dense, sparse, ultra-sparse)
|
| Types.CorrectionLocationType |
Correction location when performing operations leveraging correcting rounding
|
| Types.DataType |
Data types that can contain different ValueTypes internally.
|
| Types.Direction |
Type of aggregation direction
|
| Types.ExecMode |
Execution mode for entire script.
|
| Types.ExecType |
Execution type of individual operations.
|
| Types.FileFormat |
File formats supported
|
| Types.FunctionBlock |
Common type for both function statement blocks and function program blocks
|
| Types.OpOp1 |
Operations that require 1 operand
|
| Types.OpOp2 |
Operations that require 2 operands
|
| Types.OpOp3 |
Operations that require 3 operands
|
| Types.OpOp4 |
Operations that require 4 operands
|
| Types.OpOpData |
Data specific operations, related to reading and writing to and from memory
|
| Types.OpOpDG |
Data generation operations
|
| Types.OpOpDnn |
Deep Neural Network specific operations
|
| Types.OpOpN |
Operations that require a variable number of operands
|
| Types.ParamBuiltinOp |
Parameterized operations that require named variable arguments
|
| Types.ReOrgOp |
Operations that perform internal reorganization of an allocation
|
| Types.ReturnType |
Type of builtin or user-defined function with regard to its
number of return variables.
|
| Types.ValueType |
Value types (int, double, string, boolean, unknown).
|
| UAggOuterChain |
TODO Currently this lop only support the right hand side in distributed cache.
|
| UaggOuterChainCPInstruction |
|
| UaggOuterChainSPInstruction |
Two types of broadcast variables used -- 1.
|
| Unary |
|
| Unary |
|
| Unary |
Lop to perform following operations: with one operand -- NOT(A), ABS(A),
SQRT(A), LOG(A) with two operands where one of them is a scalar -- H=H*i,
H=H*5, EXP(A,2), LOG(A,2)
|
| UnaryCP |
|
| UnaryCPInstruction |
|
| UnaryFEDInstruction |
|
| UnaryFrameCPInstruction |
|
| UnaryFrameSPInstruction |
|
| UnaryMatrixCPInstruction |
|
| UnaryMatrixFEDInstruction |
|
| UnaryMatrixSPInstruction |
|
| UnaryOp |
|
| UnaryOperator |
|
| UnaryScalarCPInstruction |
|
| UnarySketchOperator |
|
| UnarySPInstruction |
|
| UncompressedScheme |
|
| UnifiedMemoryAllocator |
|
| UnifiedMemoryManager |
Unified Memory Manager - Initial Design
Motivation:
The Unified Memory Manager, henceforth UMM, will act as a central manager of in-memory
matrix (uncompressed and compressed), frame, and tensor blocks within SystemDS control
program.
|
| UniformPRNGenerator |
|
| Util |
|
| UtilFunctions |
|
| UtilizationModel |
|
| Utils |
|
| ValueComparisonFunction |
Abstraction for comparison (relational) operators in order to
force a proper implementation by all relevant subclasses.
|
| ValueFunction |
|
| VariableCPInstruction |
|
| VariableCPInstruction.VariableOperationCode |
|
| VariableFEDInstruction |
|
| VariableSet |
|
| VarStats |
|
| VarStats |
|
| Warnings |
|
| WeightedCell |
|
| WeightedCrossEntropy |
|
| WeightedCrossEntropy.WCeMMType |
|
| WeightedCrossEntropyR |
|
| WeightedDivMM |
|
| WeightedDivMM.WDivMMType |
|
| WeightedDivMMR |
|
| WeightedSigmoid |
|
| WeightedSigmoid.WSigmoidType |
|
| WeightedSigmoidR |
|
| WeightedSquaredLoss |
|
| WeightedSquaredLoss.WeightsType |
|
| WeightedSquaredLossR |
|
| WeightedUnaryMM |
|
| WeightedUnaryMM.WUMMType |
|
| WeightedUnaryMMR |
|
| WhileProgramBlock |
|
| WhileStatement |
|
| WhileStatementBlock |
|
| WorkerController |
|
| WorkerModel |
|
| WorkerService |
|
| WorkloadAnalyzer |
|
| WriterBinaryBlock |
|
| WriterBinaryBlockParallel |
|
| WriterCompressed |
|
| WriterHDF5 |
|
| WriterHDF5Parallel |
|
| WriterMatrixMarket |
|
| WriterMatrixMarketParallel |
|
| WriterTextCell |
|
| WriterTextCellParallel |
|
| WriterTextCSV |
|
| WriterTextCSVParallel |
|
| WriterTextLIBSVM |
|
| WriterTextLIBSVMParallel |
|
| WriteSPInstruction |
|
| WTreeNode |
A Node in the WTree, this is used for any nodes that are not the root.
|
| WTreeRoot |
The root node of the tree, located at the top of the tree.
|
| Xor |
|
| ZipmmSPInstruction |
|