This project presents a Single Input Multiple Output (SIMO) deep convolutional neural network, a so-called ART-Net (Augmented Reality Tool Network) consisting of an encoder-decoder architecture to obtain the surgical tool detection, segmentation, and geometric features concurrently in an end-to-end fashion.