-
Notifications
You must be signed in to change notification settings - Fork 1.4k
GSoC 2019 Project Ideas
Chainer is a powerful, flexible, and intuitive deep learning framework. Chainer supports CUDA computation. It only requires a few lines of code to leverage a GPU. It also runs on multiple GPUs with little effort.
ChainerX is the next generation of the Chainer code that is currently being integrated into Chainer. Based in C++, instead of Python, it provides a boost to execution speed on the CPU. It also increases extensibility for Chainer, by allowing use for different device backends, and make Chainer exportable by giving Chainer a pure C++ API.
For coding on Chainer, a solid basis in Python coding will be required. Experience working with Git and github.com will also be useful. ChainerX is written in C++, so coding experience in C++ will also be necessary.
To contact the mentors, please join the Chainer Slack workspace
Expand Chainer Primitive Operations
Expand Chainer Compiler Coverage
Assist in the updating of the Chainer documentation, which emphasis on improving the documentation within the code, such as the function and link descriptions and chains documentation.
Add additional examples to the Chainer zoo by looking at recent papers or popular approaches, and implement them in Chainer, both in the official documentation and for the examples/ directory. Current work on Chainer examples.
@Crissman
Medium to Hard
You need know:
- Python
- Machine learning code basics
- Sphinx-docs (optional)
Survey the existing functions in Chainer, and compare the documentation with documentation for other deep learning frameworks. Note what you feel is a clear explanation and inconsistencies in the Chainer documentation, then improve the Chainer documentation.
Read current papers and implement the approach from the paper in Chainer to increase the examples available for Chainer and confirm the reproducibility of the paper.
Increase the usability of a top tier machine learning framework. Deepen your own understanding of machine learning by helping explain it to others. Help users easily understand how to implement the model they’re looking for. Learn to work with self-documenting code in a large open-source project. Build relationships with coders in a top tier startup in Japan.
New primitive operations, or differentiable functions, are introduced by researchers at a fast pace. For a framework such as Chainer, it is important to implement those to make significant leaps in the field available to users. Chainer provides an interface to implement such operations with little boilerplate code but it is still difficult to keep up. Assist in expanding the support of such operations.
@beam2d, @niboshi, @hvy, @asi1024
Medium
Survey the existing operators in Chainer, and compare the feature set with recent papers or that of other multidimensional array libraries and deep learning frameworks. Find what is lacking in Chainer, and plan what you will implement for it.
Try implementing one of the operators to learn the workflow for adding a new differentiable function in Chainer.
Learn how a deep learning framework is implemented, and learn techniques for it. Find out what the latest primitive operations are and how to implement them. Build relationships with coders in a top tier startup in Japan.
ChainerX is a versatile ndarray implementation with special support of deep learning-specific operations. It therefore is important to support many fundamental operators usually available for ndarray libraries (e.g. those provided by NumPy and SciPy) as well as special operators focusing deep learning applications (e.g. convolution, pooling, activation functions, etc.). While Chainer implements many of these operators, ChainerX still has low coverage. We need full coverage of Chainer’s operators (chainer.functions) and more coverage of NumPy APIs.
Operators that are good to work on in this GSoC project include:
- Operators provided by Chainer but still lacking in ChainerX (see
chainer.functionsmodule for the full list of functions that Chainer is providing) - NumPy APIs that are lacking in ChainerX (indexing, shape/stride manipulation, math functions, linear algebra routines, statistics routines, random number generator and probability distributions, FFT, sparse matrix)
@beam2d, @niboshi, @hvy, @asi1024
Medium
You need know:
- C++
- Python
- Numpy
- Machine learning code basics (backpropagation)
Try ChainerX in both C++ and Python to learn the basic API concepts. Read through the ndarray implementation (in C++) and existing operators/routines to find its design and the how it is implemented.
Pick one of the operators from a pool of candidate operators to add. Try implementing it for the native backend (CPU) to learn the overall workflow of adding a new operator to ChainerX.
Help convert a python-based Deep Learning framework into a C++ one. Implement fast calculations on multidimensional arrays. Build relationships with coders in a top tier startup in Japan.
Recurrent neural networks (RNN) have become common in neural networks that processes series data ranging from natural language, sound to images and video. LSTM in particular is a type of module that is commonly used in these networks. Even major vendors such as NVIDIA offers a library that provides an optimized implementation of this module. RNNs are also known for being CPU heavy, often leading to significant CPU overhead when implemented in Python. For ChainerX which aims to remove this Python overhead, LSTM is a suitable module to experience the gain.
@beam2d, @niboshi, @hvy, @asi1024
Difficult
You need know:
- C++
- Python
- Numpy
- Machine learning code basics (LSTM)
Try ChainerX in both C++ and Python to learn the basic API concepts. Read through the ndarray implementation (in C++) and existing operators/routines to find its design and the how it is implemented.
Look through cuDNN documentation and learn how to use it for fast LSTM implementation. Try calling it directly and see what works and what does not. Chainer and CuPy’s existing implementation might be a good example usage to refer.
Read Chainer’s LSTM implementation (esp. L.NStepLSTM) and learn the API design and usage.
Design an appropriate API for fast LSTM in ChainerX, and write a prototype implementation in C++.
Drive Recurrent Neural Networks from Python to C++ and witness the speed improvements from your efforts. Work together in a distributed workforce to enable an additional class of accelerated Neural Networks in the Chainer framework. Build relationships with coders in a top tier startup in Japan.
ChainerX targets not only Python users, but also C++ users who need the best performance. It lacks the resources for those users, however. In this project, we would like you to enrich the C++ examples/tutorials.
@beam2d, @niboshi, @hvy, @asi1024
Easy
You need know:
- C++
- Machine Learning
- Look through Chainer Python Tutorial.
- Learn ChainerX C++ API.
- Write the tutorial. You may follow the line of existing tutorials but may also include your ideas to make it more easier to understand for ML beginners.
This project provides an opportunity to learn in-depth usage of the cutting-edge C++ machine learning framework. Learn how to write a good documentation for ML beginners. Build relationships with coders in a top tier startup in Japan.
As Python is the most used language in machine learning researches, ChainerX is equipped with built-in Python binding. Its core is written in C++ for optimized speed, however.
As the field of deep learning is expanding from researches to deployment, we would like to support other languages as well. In this project, we would like you to implement the binding of another language than Python, and pave the road for the language’s users to benefit from the fast deep learning framework.
@beam2d, @niboshi, @hvy, @asi1024
Medium to Hard
You need know:
- C and C++
- Any other language that you want to write a binding for, e.g. Rust, Go, JavaScript (Node.js), Ruby, C#, Java, Scala, Swift.
- Decide the language for which you want to write the binding. Consider the typical use-cases of deep learning using that language, as well as the population of users.
- Learn ChainerX C++ API.
- Check out ChainerX Python Examples.
- Learn how to write a binding in the language you decided.
- Provides an opportunity to learn the basics of interop between languages.
- Increase the coverage of a fast ML framework
- Expand the tools available to the language you want to write a binding for
- Build relationships with coders in a top tier startup in Japan.
Chainer-compiler is an experimental compiler stack to make chainer models deployable and run faster. Since the project is fairly young, its coverage is not as good as Chainer’s. This project spans a bunch of areas. You would choose one of these areas and expand the coverage of the area. Example goals are
- Map more Python functions to ONNX (e.g., chainer/functions/activation)
- Map ONNX ops to ChainerX routines (e.g., Tile)
- Define gradient for ops
- Implement complex Python feature such as break
- Run BERT/Transformer with chainer-compiler
- Translate ONNX model to TensorRT (or other NN backends)
but we could find other goals together.
@shinh
Medium to Hard
The difficulty depends on the theme you will work on. The first three example topics wouldn’t be hard, but the other three topics (Python features, BERT, and TensorRT) would be challenging and probably more fun.
Prerequisites also depend on your interest. Example requirements are
- Python or C++
- Chainer or ONNX
- Machine learning code basics
- Learn how chainer compiler works. We should admit the code is not easy to understand and we have almost no documents. Please feel free to ask any questions!
- Find an area you are interested in. Again, feel free to contact us. If you tell us your background, we could suggest a few example topics.
- Learn Python AST, ONNX, or ChainerX, depending your topic.
Learn the structure of a compiler stack which spans on broad computer science areas. Choose which part of the compiler stack to work on. Build relationships with coders in a top tier startup in Japan.