Data Consultants near you

A warehouse stores structured, modelled data: the shape is decided before loading, which makes querying fast and reporting consistent, and makes every new source a piece of design work. A lake stores raw files in their original form, which is cheap and accommodating and puts the interpretation burden on whoever queries it later. Neither is inherently right.

Most UK mid-sized organisations are better served by a warehouse with clear definitions than by a lake nobody curates, because their problem is disagreement about numbers rather than volume of data. Lakes earn their place with large volumes of semi-structured data, machine learning work, or sources whose structure changes often. A consultant who proposes the architecture before asking what questions the business needs answered has chosen the tool first.

Lineage is the record of how a figure in a dashboard traces back through transformations to the rows in the source system. Without it, a disagreement about a number becomes an archaeology project, and nobody can tell whether a change to an upstream system will break a report. With it, the question is answerable in minutes.

Ask for lineage as a deliverable, in whatever form the tooling supports, alongside a definitions document that says in plain English what each measure means, which records it includes and excludes, and who owns the definition. That document is what stops two departments presenting different revenue figures at the same meeting, which is the outcome most data projects were commissioned to prevent.

The ICO's guidance treats identifiability as a spectrum rather than a switch, and uses a motivated intruder test: whether a reasonably competent person, with reasonable resources and a motivation to try, could re-identify individuals in the dataset. Replacing names with reference numbers is pseudonymisation, and pseudonymised data remains personal data because the link can be restored.

The practical consequence for an analytics project is that a dataset described as anonymised usually is not, and the data protection obligations continue to apply to it. Small-group analysis is where this bites: a count of one employee in one location on one date identifies someone regardless of whether a name is present. Agree thresholds for suppressing small counts, review identifiability when new sources are added, and record the reasoning so the assessment can be revisited rather than reinvented.

The ICO publishes a data sharing code of practice intended to help organisations share personal data fairly, safely and transparently, covering data sharing agreements, accountability, lawfulness, transparency, security and individual rights, with checklists and case studies. It is the reference point when your data project starts exchanging records with another organisation rather than simply reorganising your own.

A written data sharing agreement should state what is shared, why, on what legal basis, for how long, what security applies and who answers a request from an individual. Where a supplier processes data for you rather than for its own purposes, you need a processor contract instead, and analytics consultants who take extracts to their own environment are squarely in that category. Say in writing where copies may live and when they are destroyed.

Before a data project, an attacker who reaches one system gets one system's worth of data. Afterwards, a single set of credentials may reach the combined history of every customer. The ICO expects a DPIA where processing is likely to result in high risk and lists data matched or combined from multiple sources among its indicators, so this concentration is exactly the kind of change that ought to trigger an assessment during design.

The controls are unglamorous: role-based access so analysts see only the columns they need, masking of direct identifiers in the default views, separate credentials for pipelines and for people, logging of who queried what, and a retention rule that removes source-level detail once the aggregates are what the business uses. If a breach did happen, the difference between reporting an incident involving identifiable full histories and one involving aggregated figures is the difference these controls buy.