SIGN IN SIGN UP
apache / mxnet UNCLAIMED

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

Fix PR #15489 (Dynamic Library Loading Support) (#15760) * Dynamic Library Loading Support (#15489) * Accelerator APIs header file * adding example to test accelerator loading * adding c_api function header * modifying header file path * creating templates for call to lib fns * modifying target to mxnet_static for libdl * rebaseing with master * returning nullptr handle if library not loaded * refactoring code to load libraries dynamically * addressing review comments * using static cast * pylint fix * moving library.h file to src/common/ * adding dynamic loading support for windows * updating header guard * fixing headers * fixing windows casting error * declaring library functions for windows * adding library testing module in examples * adding unit test to test library loading * correcting file names * updating error messages * getting error message from DL library * adding unit test to gpu suite * correcting windows pointer * requiring absolute path to library * changing file description * addressing review comments - adding more docs, windows error msg * addressing PR comments * checking machine type for unit test * “re-trigger” * added map to store loaded libraries * added dlclose calls in naive & threaded engines * removed library map declaration in cc file * added windows free * fixed formatting * added cast to HMODULE for void* for windows * retrigger CI for flaky unix_cpu * build library and stash it in CI * modifying unittest to use CI built binary * Retrigger CI * adding dlclose to initialize destructor * adding sample_lib target to all in Makefile
2019-08-07 17:34:29 -07:00
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
/*!
* \file libtest.cc
* \brief This test checks if the library is implemented correctly
* and does not involve dynamic loading of library into MXNet
* This test is supposed to be run before test.py
*/
#if defined(_WIN32) || defined(_WIN64) || defined(__WINDOWS__)
#include <windows.h>
#else
#include <dlfcn.h>
#endif
#include <iostream>
#include "lib_api.h"
#define MXNET_VERSION 10500
int main(void) {
// Get a handle to the library.
#if defined(_WIN32) || defined(_WIN64) || defined(__WINDOWS__)
HINSTANCE handle;
dynamic custom operator support (#15921) * fixed example to use absolute path * added example for custom ops, added support for custom op registration * added fcompute registration for loaded operators moved library import order to after ndarray/symbol * changed dynamic ops to be contrib * added num in/out * removed contrib op registration re-registered ops from mx.nd.op to mx.nd * added support for infer shape, updated example to call operator * fixed whitespace * fixed whitespace * fixed whitespace * added temporary support for operator multi-registration * insanity checked * update docblocks * small format fix * fix unittest with correct library * implement InferType * initial support for resource manager, temp space * fixed formatting * changed decltype to typedef * fixed whitespace * Added windows declaration types, change APIs to return MXReturnValue instead of int * added library version number, API to get, and check to validate * Changed CMakeLists to build lib_ops instead of lib_api, updated lib_api example, fixed whitespace * add prototype of subgraph op * implement FMutateInput as optional attribute * fix sanity check * replace fcompute to fcomputeEx and implement simple finferstoragetype * changed fcompute to forward * initial commit with fgradient support * enabled gradient registration * fixed whitespace * fixed example to use absolute path * added example for custom ops, added support for custom op registration * added fcompute registration for loaded operators moved library import order to after ndarray/symbol * changed dynamic ops to be contrib * added num in/out * removed contrib op registration re-registered ops from mx.nd.op to mx.nd * added support for infer shape, updated example to call operator * fixed whitespace * fixed whitespace * fixed whitespace * added temporary support for operator multi-registration * insanity checked * update docblocks * small format fix * fix unittest with correct library * implement InferType * initial support for resource manager, temp space * fixed formatting * changed decltype to typedef * fixed whitespace * Added windows declaration types, change APIs to return MXReturnValue instead of int * added library version number, API to get, and check to validate * Changed CMakeLists to build lib_ops instead of lib_api, updated lib_api example, fixed whitespace * add prototype of subgraph op * implement FMutateInput as optional attribute * fix sanity check * replace fcompute to fcomputeEx and implement simple finferstoragetype * changed fcompute to forward * initial commit with fgradient support * enabled gradient registration * fixed whitespace * prototype of createopstate and fstatefulcompute * make custom state op interface work * subgraph forward * refactor stateful forward and add op resource * wip gemm backward * stateful backward and subgraph test * implement gemm and state gemm, refactor test files * add body to pure virtual destructor * subgraph passing from python to custom lib * rm lib_api c++11 dep, rm warpctc, add rm flag * fix conflict * subgraph json parsing utility * add data size and fix unsigned warnings * use c++ struct and fix cpplint * refactor op registry * fix line length and win array of ci; condense lines * add mxnet_extension dir * fixed extension to be dll for windows * updated examples to use the same format as the example in the top-level Makefile: "lib<name>.so" * removed destructor for CustomStatefulOp * fix error in gemm test and clear up subgraph test * lib path fix * add unittest for custom op * update Makefile revolve merge * fix test and rename folder * fix makefile rename * fix cmake rename * add explicit cpu context * wkcn feedback: change mxtensor func name. use c++11 flag * add operator keyward test and refine info print * using typedef in forward * small refine of docblock * change names * add separate stateful compute and pass state_op ptr * user example using opresource alloc * added DLTensor into MXTensor * fixed whitespace * added error check when DLTensor does not support MXNet data type * changed to throw runtime exception * changed include to stdexcept * retrigger CI * empty commit * empty commit * remove merge conflict * add setdltensor for easy use and add docs * CI * re-trigger CI * ci * ci
2019-12-05 21:34:47 -08:00
handle = LoadLibrary(TEXT("libinit_lib.dll"));
Fix PR #15489 (Dynamic Library Loading Support) (#15760) * Dynamic Library Loading Support (#15489) * Accelerator APIs header file * adding example to test accelerator loading * adding c_api function header * modifying header file path * creating templates for call to lib fns * modifying target to mxnet_static for libdl * rebaseing with master * returning nullptr handle if library not loaded * refactoring code to load libraries dynamically * addressing review comments * using static cast * pylint fix * moving library.h file to src/common/ * adding dynamic loading support for windows * updating header guard * fixing headers * fixing windows casting error * declaring library functions for windows * adding library testing module in examples * adding unit test to test library loading * correcting file names * updating error messages * getting error message from DL library * adding unit test to gpu suite * correcting windows pointer * requiring absolute path to library * changing file description * addressing review comments - adding more docs, windows error msg * addressing PR comments * checking machine type for unit test * “re-trigger” * added map to store loaded libraries * added dlclose calls in naive & threaded engines * removed library map declaration in cc file * added windows free * fixed formatting * added cast to HMODULE for void* for windows * retrigger CI for flaky unix_cpu * build library and stash it in CI * modifying unittest to use CI built binary * Retrigger CI * adding dlclose to initialize destructor * adding sample_lib target to all in Makefile
2019-08-07 17:34:29 -07:00
#else
void* handle;
handle = dlopen("libinit_lib.so", RTLD_LAZY);
Fix PR #15489 (Dynamic Library Loading Support) (#15760) * Dynamic Library Loading Support (#15489) * Accelerator APIs header file * adding example to test accelerator loading * adding c_api function header * modifying header file path * creating templates for call to lib fns * modifying target to mxnet_static for libdl * rebaseing with master * returning nullptr handle if library not loaded * refactoring code to load libraries dynamically * addressing review comments * using static cast * pylint fix * moving library.h file to src/common/ * adding dynamic loading support for windows * updating header guard * fixing headers * fixing windows casting error * declaring library functions for windows * adding library testing module in examples * adding unit test to test library loading * correcting file names * updating error messages * getting error message from DL library * adding unit test to gpu suite * correcting windows pointer * requiring absolute path to library * changing file description * addressing review comments - adding more docs, windows error msg * addressing PR comments * checking machine type for unit test * “re-trigger” * added map to store loaded libraries * added dlclose calls in naive & threaded engines * removed library map declaration in cc file * added windows free * fixed formatting * added cast to HMODULE for void* for windows * retrigger CI for flaky unix_cpu * build library and stash it in CI * modifying unittest to use CI built binary * Retrigger CI * adding dlclose to initialize destructor * adding sample_lib target to all in Makefile
2019-08-07 17:34:29 -07:00
#endif
if (!handle) {
std::cerr << "Unable to load library" << std::endl;
return 1;
}
// get initialize function address from the library
initialize_t init_lib;
#if defined(_WIN32) || defined(_WIN64) || defined(__WINDOWS__)
init_lib = (initialize_t)GetProcAddress(handle, MXLIB_INITIALIZE_STR);
Fix PR #15489 (Dynamic Library Loading Support) (#15760) * Dynamic Library Loading Support (#15489) * Accelerator APIs header file * adding example to test accelerator loading * adding c_api function header * modifying header file path * creating templates for call to lib fns * modifying target to mxnet_static for libdl * rebaseing with master * returning nullptr handle if library not loaded * refactoring code to load libraries dynamically * addressing review comments * using static cast * pylint fix * moving library.h file to src/common/ * adding dynamic loading support for windows * updating header guard * fixing headers * fixing windows casting error * declaring library functions for windows * adding library testing module in examples * adding unit test to test library loading * correcting file names * updating error messages * getting error message from DL library * adding unit test to gpu suite * correcting windows pointer * requiring absolute path to library * changing file description * addressing review comments - adding more docs, windows error msg * addressing PR comments * checking machine type for unit test * “re-trigger” * added map to store loaded libraries * added dlclose calls in naive & threaded engines * removed library map declaration in cc file * added windows free * fixed formatting * added cast to HMODULE for void* for windows * retrigger CI for flaky unix_cpu * build library and stash it in CI * modifying unittest to use CI built binary * Retrigger CI * adding dlclose to initialize destructor * adding sample_lib target to all in Makefile
2019-08-07 17:34:29 -07:00
#else
init_lib = (initialize_t)dlsym(handle, MXLIB_INITIALIZE_STR);
Fix PR #15489 (Dynamic Library Loading Support) (#15760) * Dynamic Library Loading Support (#15489) * Accelerator APIs header file * adding example to test accelerator loading * adding c_api function header * modifying header file path * creating templates for call to lib fns * modifying target to mxnet_static for libdl * rebaseing with master * returning nullptr handle if library not loaded * refactoring code to load libraries dynamically * addressing review comments * using static cast * pylint fix * moving library.h file to src/common/ * adding dynamic loading support for windows * updating header guard * fixing headers * fixing windows casting error * declaring library functions for windows * adding library testing module in examples * adding unit test to test library loading * correcting file names * updating error messages * getting error message from DL library * adding unit test to gpu suite * correcting windows pointer * requiring absolute path to library * changing file description * addressing review comments - adding more docs, windows error msg * addressing PR comments * checking machine type for unit test * “re-trigger” * added map to store loaded libraries * added dlclose calls in naive & threaded engines * removed library map declaration in cc file * added windows free * fixed formatting * added cast to HMODULE for void* for windows * retrigger CI for flaky unix_cpu * build library and stash it in CI * modifying unittest to use CI built binary * Retrigger CI * adding dlclose to initialize destructor * adding sample_lib target to all in Makefile
2019-08-07 17:34:29 -07:00
#endif
if (!init_lib) {
std::cerr << "Unable to get function 'intialize' from library" << std::endl;
return 1;
}
// Call the function.
(init_lib)(MXNET_VERSION);
// Deallocate memory.
#if defined(_WIN32) || defined(_WIN64) || defined(__WINDOWS__)
FreeLibrary(handle);
#else
dlclose(handle);
#endif
return 0;
}