Diversity, in every dimension, is a key attribute of today’s data bonanza. Our research takes a holistic view, embracing this diversity and the consequent intricate interactions between users and systems. We created the Dispel data-streaming language to describe complex computation patterns at high levels of abstraction, while providing meta-information for optimisation. Provenance and contextual information must be harnessed to achieve autonomous execution, data placement, energy efficiency and reliability. Research is needed, that builds foundations for comprehensible and tractable paths through the rapidly evolving data landscape. This must be driven by and tested at realistic scales and diversity levels.