While there have been huge advances in Machine Learning in recent years, many of the successes have relied on immense amounts of training data. This talk focuses on such Human-in-the-Loop Machine Learning for robotics tasks, covering both navigations, especially in tightly constrained spaces, and manipulation in open-world settings.