Blame: example/module/sequential_module.py - apache/mxnet

apache / mxnet UNCLAIMED

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

0 0 0 C++

Normal View History Raw

Add license header (#7379) * add * add .py and ci * fix pylint * update 2017-08-08 16:36:23 -07:00			`# Licensed to the Apache Software Foundation (ASF) under one`
			`# or more contributor license agreements. See the NOTICE file`
			`# distributed with this work for additional information`
			`# regarding copyright ownership. The ASF licenses this file`
			`# to you under the Apache License, Version 2.0 (the`
			`# "License"); you may not use this file except in compliance`
			`# with the License. You may obtain a copy of the License at`
			`#`
			`# http://www.apache.org/licenses/LICENSE-2.0`
			`#`
			`# Unless required by applicable law or agreed to in writing,`
			`# software distributed under the License is distributed on an`
			`# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY`
			`# KIND, either express or implied. See the License for the`
			`# specific language governing permissions and limitations`
			`# under the License.`

sequential module container 2016-04-13 12:54:38 -04:00			`# pylint: skip-file`
fix example (#4210) * fix example * fix * fix 2016-12-20 21:29:32 -08:00			`import os, sys`
			`from utils import get_data`
sequential module container 2016-04-13 12:54:38 -04:00			`import mxnet as mx`
			`import numpy as np`
			`import logging`

a very simple model-par + data-par example 2016-04-14 18:56:02 -04:00			`# whether to demo model-parallelism + data parallelism`
Time major data layout support for RNN (#3497) * rnn-cell demo (push to server for testing) * a running example with cuDNN RNN cell * ndarray concatenate * fix lint errors * allow batch_axis in executor_group * add batch_axis parameter for all modules * fix bug in copy slice implementation * fix module examples * use batch_axis if data iterator provided such information * rnn cell example in time major * fix init state names in rnn cell bucketing example * sanity check stochastic depth mnist * a cifar10 example (not tested) * add description for sd cifar * add doc for sd module * add a simple random number queue * add final numbers * fix typo * default layout mapper * fix other modules for layout mapper * fix typo * softmax output mode that preserves the shape * comments on run-time speed of time-major * extend layout mapper to include other information * fix data layout API change * fix lint errors * fix Travis CI numpy error on unit test 2016-10-13 22:54:01 -04:00			`demo_data_model_parallelism = True`
a very simple model-par + data-par example 2016-04-14 18:56:02 -04:00
			`if demo_data_model_parallelism:`
			`contexts = [[mx.context.gpu(0), mx.context.gpu(1)], [mx.context.gpu(2), mx.context.gpu(3)]]`
			`else:`
			`contexts = [mx.context.cpu(), mx.context.cpu()]`

sequential module container 2016-04-13 12:54:38 -04:00			`#--------------------------------------------------------------------------------`
			`# module 1`
			`#--------------------------------------------------------------------------------`
			`data = mx.symbol.Variable('data')`
			`fc1 = mx.symbol.FullyConnected(data, name='fc1', num_hidden=128)`
			`act1 = mx.symbol.Activation(fc1, name='relu1', act_type="relu")`

a very simple model-par + data-par example 2016-04-14 18:56:02 -04:00			`mod1 = mx.mod.Module(act1, label_names=[], context=contexts[0])`
sequential module container 2016-04-13 12:54:38 -04:00
			`#--------------------------------------------------------------------------------`
			`# module 2`
			`#--------------------------------------------------------------------------------`
auto-wiring of modules 2016-04-13 14:13:23 -04:00			`data = mx.symbol.Variable('data')`
sequential module container 2016-04-13 12:54:38 -04:00			`fc2 = mx.symbol.FullyConnected(data, name = 'fc2', num_hidden = 64)`
			`act2 = mx.symbol.Activation(fc2, name='relu2', act_type="relu")`
			`fc3 = mx.symbol.FullyConnected(act2, name='fc3', num_hidden=10)`
			`softmax = mx.symbol.SoftmaxOutput(fc3, name = 'softmax')`

a very simple model-par + data-par example 2016-04-14 18:56:02 -04:00			`mod2 = mx.mod.Module(softmax, context=contexts[1])`
sequential module container 2016-04-13 12:54:38 -04:00
			`#--------------------------------------------------------------------------------`
			`# Container module`
			`#--------------------------------------------------------------------------------`
			`mod_seq = mx.mod.SequentialModule()`
auto-wiring of modules 2016-04-13 14:13:23 -04:00			`mod_seq.add(mod1).add(mod2, take_labels=True, auto_wiring=True)`
sequential module container 2016-04-13 12:54:38 -04:00

			`#--------------------------------------------------------------------------------`
			`# Training`
			`#--------------------------------------------------------------------------------`
			`n_epoch = 2`
			`batch_size = 100`
Time major data layout support for RNN (#3497) * rnn-cell demo (push to server for testing) * a running example with cuDNN RNN cell * ndarray concatenate * fix lint errors * allow batch_axis in executor_group * add batch_axis parameter for all modules * fix bug in copy slice implementation * fix module examples * use batch_axis if data iterator provided such information * rnn cell example in time major * fix init state names in rnn cell bucketing example * sanity check stochastic depth mnist * a cifar10 example (not tested) * add description for sd cifar * add doc for sd module * add a simple random number queue * add final numbers * fix typo * default layout mapper * fix other modules for layout mapper * fix typo * softmax output mode that preserves the shape * comments on run-time speed of time-major * extend layout mapper to include other information * fix data layout API change * fix lint errors * fix Travis CI numpy error on unit test 2016-10-13 22:54:01 -04:00			`basedir = os.path.dirname(__file__)`
fix example (#4210) * fix example * fix * fix 2016-12-20 21:29:32 -08:00			`get_data.get_mnist(os.path.join(basedir, "data"))`

sequential module container 2016-04-13 12:54:38 -04:00			`train_dataiter = mx.io.MNISTIter(`
fix example (#4210) * fix example * fix * fix 2016-12-20 21:29:32 -08:00			`image=os.path.join(basedir, "data", "train-images-idx3-ubyte"),`
			`label=os.path.join(basedir, "data", "train-labels-idx1-ubyte"),`
sequential module container 2016-04-13 12:54:38 -04:00			`data_shape=(784,),`
			`batch_size=batch_size, shuffle=True, flat=True, silent=False, seed=10)`
			`val_dataiter = mx.io.MNISTIter(`
fix example (#4210) * fix example * fix * fix 2016-12-20 21:29:32 -08:00			`image=os.path.join(basedir, "data", "t10k-images-idx3-ubyte"),`
			`label=os.path.join(basedir, "data", "t10k-labels-idx1-ubyte"),`
sequential module container 2016-04-13 12:54:38 -04:00			`data_shape=(784,),`
			`batch_size=batch_size, shuffle=True, flat=True, silent=False)`

			`logging.basicConfig(level=logging.DEBUG)`
			`mod_seq.fit(train_dataiter, eval_data=val_dataiter,`
			`optimizer_params={'learning_rate':0.01, 'momentum': 0.9}, num_epoch=n_epoch)`