com.yahoo.ml.caffe

ImageDataSource

abstract class ImageDataSource extends DataSource[(String, String, Int, Int, Int, Boolean, Array[Byte]), MatVector]

abstract image data source for images. Its subclass should construct RDD of ((id: String, label: String), value : byte[]).

Linear Supertypes
DataSource[(String, String, Int, Int, Int, Boolean, Array[Byte]), MatVector], Serializable, Serializable, AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. ImageDataSource
  2. DataSource
  3. Serializable
  4. Serializable
  5. AnyRef
  6. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ImageDataSource(conf: Config, layerId: Int, isTrain: Boolean)

    conf

    CaffeSpark configuration

    layerId

    the layer index in the network protocol file

    isTrain

Abstract Value Members

  1. abstract def makeRDD(sc: SparkContext): RDD[(String, String, Int, Int, Int, Boolean, Array[Byte])]

    construct a sample RDD

    construct a sample RDD

    sc

    spark context

    returns

    RDD created from this source

    Definition Classes
    DataSource

Concrete Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. val STOP_MARK: (String, String, Int, Int, Int, Boolean, Array[Byte])

    Definition Classes
    DataSource
  7. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  8. def batchSize(): Int

    batch size

    batch size

    Definition Classes
    DataSource
  9. var batchSize_: Int

    Attributes
    protected
    Definition Classes
    DataSource
  10. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  11. def dummyDataBlobs(): Array[FloatBlob]

    make a dummy data blob to be used by Solver threads

    make a dummy data blob to be used by Solver threads

    returns

    a dummy data blob

    Definition Classes
    ImageDataSourceDataSource
  12. def dummyDataHolder(): MatVector

    make a dummy data blob to be used by Solver threads

    make a dummy data blob to be used by Solver threads

    returns

    a dummy data blob

    Definition Classes
    ImageDataSourceDataSource
  13. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  14. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  15. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  16. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  17. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  18. def init(): Boolean

    initialization of a Source within a process

    initialization of a Source within a process

    returns

    true if successfully initialized

    Definition Classes
    ImageDataSourceDataSource
  19. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  20. var log: Logger

    Attributes
    protected
  21. var memdatalayer_param: MemoryDataParameter

    Attributes
    protected
  22. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  23. def nextBatch(sampleIds: Array[String], mats: MatVector, labels: FloatBlob): Boolean

    create a batch of samples extracted from source queue

    create a batch of samples extracted from source queue

    This method is Invoked by Transformer thread. You should extract samples from source queue, parse it and produce a batch.

    sampleIds

    holder for sample Ids

    labels

    holder for label blob

    returns

    true if successful

    Definition Classes
    ImageDataSourceDataSource
  24. final def notify(): Unit

    Definition Classes
    AnyRef
  25. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  26. def setBatchSize(size: Int): Unit

    adjust batch size

    adjust batch size

    size

    the new batch size

    Definition Classes
    DataSource
  27. var solverMode: Int

    Attributes
    protected
    Definition Classes
    DataSource
  28. var sourceFilePath: String

    Attributes
    protected
    Definition Classes
    DataSource
  29. var sourceQueue: ArrayBlockingQueue[(String, String, Int, Int, Int, Boolean, Array[Byte])]

    Definition Classes
    DataSource
  30. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  31. def toString(): String

    Definition Classes
    AnyRef → Any
  32. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  33. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  34. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from DataSource[(String, String, Int, Int, Int, Boolean, Array[Byte]), MatVector]

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped