2017-07-21 21:37:30 -07:00
# Image API
## Overview
This document summarizes supporting functions and iterators to read and process
images provided in
``` eval_rst
.. autosummary::
:nosignatures:
mxnet.image
```
## Image processing functions
``` eval_rst
.. currentmodule:: mxnet
.. autosummary::
:nosignatures:
image.imdecode
image.scale_down
image.resize_short
image.fixed_crop
image.random_crop
image.center_crop
image.color_normalize
image.random_size_crop
```
## Image iterators
Iterators support loading image from binary `Record IO` and raw image files.
```eval_rst
.. autosummary::
:nosignatures:
image.ImageIter
```
``` python
>> > data_iter = mx . image . ImageIter ( batch_size = 4 , data_shape = ( 3 , 224 , 224 ) , label_width = 1 ,
path_imglist = ' data/custom.lst ' )
>> > data_iter . reset ( )
>> > for data in data_iter :
. . . d = data . data [ 0 ]
. . . print ( d . shape )
>> > # we can apply lots of augmentations as well
>> > data_iter = mx . image . ImageIter ( 4 , ( 3 , 224 , 224 ) , path_imglist = ' data/custom.lst ' ,
2018-03-20 23:00:34 -04:00
rand_crop = True , rand_resize = True , rand_mirror = True , mean = True ,
2017-07-21 21:37:30 -07:00
brightness = 0.1 , contrast = 0.1 , saturation = 0.1 , hue = 0.1 ,
pca_noise = 0.1 , rand_gray = 0.05 )
>> > data = data_iter . next ( )
>> > # specify augmenters manually is also supported
>> > data_iter = mx . image . ImageIter ( 32 , ( 3 , 224 , 224 ) , path_rec = ' data/caltech.rec ' ,
path_imgidx = ' data/caltech.idx ' , shuffle = True ,
aug_list = [ mx . image . HorizontalFlipAug ( 0.5 ) ,
mx . image . ColorJitterAug ( 0.1 , 0.1 , 0.1 ) ] )
```
We use helper function to initialize augmenters
``` eval_rst
.. currentmodule:: mxnet
.. autosummary::
:nosignatures:
image.CreateAugmenter
```
A list of supporting augmenters
``` eval_rst
.. autosummary::
:nosignatures:
image.Augmenter
2017-10-13 12:48:37 -07:00
image.SequentialAug
image.RandomOrderAug
2017-07-21 21:37:30 -07:00
image.ResizeAug
image.ForceResizeAug
image.RandomCropAug
image.RandomSizedCropAug
image.CenterCropAug
image.BrightnessJitterAug
image.ContrastJitterAug
image.SaturationJitterAug
image.HueJitterAug
image.ColorJitterAug
image.LightingAug
image.ColorNormalizeAug
image.RandomGrayAug
image.HorizontalFlipAug
image.CastAug
```
Similar to `ImageIter` , `ImageDetIter` is designed for `Object Detection` tasks.
``` eval_rst
.. autosummary::
:nosignatures:
image.ImageDetIter
```
``` python
>> > data_iter = mx . image . ImageDetIter ( batch_size = 4 , data_shape = ( 3 , 224 , 224 ) ,
path_imglist = ' data/train.lst ' )
>> > data_iter . reset ( )
>> > for data in data_iter :
. . . d = data . data [ 0 ]
. . . l = data . label [ 0 ]
. . . print ( d . shape )
. . . print ( l . shape )
```
Unlike object classification with fixed label_width, object count may vary from
image to image. Thus we have special format for object detection labels.
Usually the `lst` file generated by `tools/im2rec.py` is a list of
```
index_0 label_0 image_path_0
index_1 label_1 image_path_1
```
Where `label_N` is a number a of fixed-width vector.
The format of label used in object detection is a variable length vector
```
2017-09-06 15:18:22 -07:00
A B [extra header] [(object0), (object1), ... (objectN)]
2017-07-21 21:37:30 -07:00
```
2017-09-06 15:18:22 -07:00
Where A is the width of header (2 + length of extra header), B is the width of each object.
Extra header is optional and used for inserting helper information such as (width, height).
2017-07-21 21:37:30 -07:00
Each object is usually 5 or 6 numbers describing the object properties, for example:
[id, xmin, ymin, xmax, ymax, difficulty]
Putting all together, we have a `lst` file for object detection:
```
2017-09-06 15:18:22 -07:00
0 4 5 640 480 1 0.1 0.2 0.8 0.9 2 0.5 0.3 0.6 0.8 data/xxx.jpg
1 4 5 480 640 3 0.05 0.16 0.75 0.9 data/yyy.jpg
2 4 5 500 600 2 0.6 0.1 0.7 0.5 0 0.1 0.3 0.2 0.4 3 0.25 0.25 0.3 0.3 data/zzz.jpg
2017-07-21 21:37:30 -07:00
...
```
A helper function to initialize Augmenters for `Object detection` task
``` eval_rst
.. autosummary::
:nosignatures:
image.CreateDetAugmenter
```
Since `Detection` task is sensitive to object localization, any modification
to image that introduced localization shift will require correction to label,
and a list of augmenters specific for `Object detection` is provided
```eval_rst
.. autosummary::
:nosignatures:
image.DetBorrowAug
image.DetRandomSelectAug
image.DetHorizontalFlipAug
image.DetRandomCropAug
image.DetRandomPadAug
```
## API Reference
<script type="text/javascript" src='../../_static/js/auto_module_index.js'></script>
``` eval_rst
.. automodule:: mxnet.image
.. autoclass:: mxnet.image.ImageIter
:members:
.. automethod:: mxnet.image.imdecode
.. automethod:: mxnet.image.scale_down
.. automethod:: mxnet.image.resize_short
.. automethod:: mxnet.image.fixed_crop
.. automethod:: mxnet.image.random_crop
.. automethod:: mxnet.image.center_crop
.. automethod:: mxnet.image.color_normalize
.. automethod:: mxnet.image.random_size_crop
.. autoclass:: mxnet.image.Augmenter
:members:
.. autoclass:: mxnet.image.ResizeAug
.. autoclass:: mxnet.image.ForceResizeAug
.. autoclass:: mxnet.image.RandomCropAug
.. autoclass:: mxnet.image.RandomSizedCropAug
.. autoclass:: mxnet.image.CenterCropAug
.. autoclass:: mxnet.image.RandomOrderAug
.. autoclass:: mxnet.image.BrightnessJitterAug
.. autoclass:: mxnet.image.ContrastJitterAug
.. autoclass:: mxnet.image.SaturationJitterAug
.. autoclass:: mxnet.image.HueJitterAug
.. autoclass:: mxnet.image.ColorJitterAug
.. autoclass:: mxnet.image.LightingAug
.. autoclass:: mxnet.image.ColorNormalizeAug
.. autoclass:: mxnet.image.RandomGrayAug
.. autoclass:: mxnet.image.HorizontalFlipAug
.. autoclass:: mxnet.image.CastAug
.. automethod:: mxnet.image.CreateAugmenter
.. autoclass:: mxnet.image.ImageDetIter
:members:
.. autoclass:: mxnet.image.DetAugmenter
:members:
.. autoclass:: mxnet.image.DetBorrowAug
.. autoclass:: mxnet.image.DetRandomSelectAug
.. autoclass:: mxnet.image.DetHorizontalFlipAug
.. autoclass:: mxnet.image.DetRandomCropAug
.. autoclass:: mxnet.image.DetRandomPadAug
.. automethod:: mxnet.image.CreateDetAugmenter
```
<script>auto_index("api-reference");</script>
