In recent years, reduction of energy consumption in buildings has increasingly gained interest among researchers mainly due to practical reasons, such as economic advantages and long-term environmental sustainability. Many solutions have been proposed in the literature to address this important issue from complementary perspectives, which are often hard to capture in a comprehensive manner. This survey article aims at providing a structured and unifying treatment of the existing literature on intelligent energy management systems in buildings, with a distinct focus on available architectures and methodology supporting a vision transcending the well-established smart home vision, in favor of the novel Ambient Intelligence paradigm. Our exposition will cover the main architectural components of such systems, beginning with the basic sensory infrastructure, moving on to the data processing engine where energy-saving strategies may be enacted, to the user interaction interface subsystem, and finally to the actuation infrastructure necessary to transfer the planned modifications to the environment. For each component, we will analyze different solutions, and we will provide qualitative comparisons, also highlighting the impact that a single design choice can have on the rest of the system.