Deep neural networks have revolutionized Machine Learning, completely reshaping several domains of research in a mere decade. Domains like computer vision, for which deep learning outclassed the previous approaches based on handcrafted features. For instance, to the current day (end of 2021), the most baseline approach to extract features from a RGB image is to use a neural network trained on the popular Image classification task ImageNet. More generally, in the most popular domains, there is a great deal of literature, good practices, and pretrained models accessible with a few lines of Python code. However, not all problems are as crowded as computer vision.

In some cases, a single laboratory can deal with multiple types of sensors (accelerometers, strain sensors, GPS signals, physiological signals) to perform Machine Learning on, in the span of a few months. For many of these unnoticed tasks, deep neural networks require considerable work: type of preprocessing, hyperparameter selection, choice of an encoding, or even sensor choice, depending on the problem. We will take the place of a practitioner and review the most common choices we would have to make in order to make deep neural networks work with temporal signals. In each case, we will give indications the best choice to make, according to our experiments and/or the literature. Most of our experiments will focus on Transport Mode Detection, but we will use the literature to distinguish between the conclusions that only apply to our problem, and the affirmations that generalize elsewhere.