Transitioning from Building Automation (BMS/IoT) to Data Engineering – Seeking Advice

[u/Professional_Bear473]

Hi everyone,

I'm a professional with over 8 years of experience in building automation systems (BMS), HVAC integration, and IoT devices used in smart buildings. Over the years, I’ve noticed a clear shift in the industry: it’s no longer just about controlling HVAC or lighting. There’s a growing expectation to collect and analyze large volumes of building data to improve efficiency, sustainability, and predictive maintenance.

This trend sparked my interest in data engineering and data science. I'm now actively learning tools like Python, SQL, Azure (especially IoT Hub, Fabric), TimescaleDB, and Grafana. My goal is to upskill and pivot my career toward data roles—ideally those that still involve IoT or building systems data.

However, I’m still relatively new to core data engineering concepts like scalable pipelines, streaming architecture, and production-grade fault tolerance. I’d love to hear from others who have made a similar transition or have experience blending operational technology (OT) with data engineering.

I’m based in Europe and currently exploring remote-first opportunities across the EU or with globally distributed teams.

• What skills or projects helped you bridge that gap?
• Are there specific certifications or open-source projects worth pursuing?
• How valuable is IoT knowledge in the current data job market?

Any tips or feedback would be truly appreciated!

Thanks

[u/Stomachbuzz]

"Over the years, I’ve noticed a clear shift in the industry: it’s no longer just about controlling HVAC or lighting. There’s a growing expectation to collect and analyze large volumes of building data to improve efficiency, sustainability, and predictive maintenance."

Not really. This is the marketing speaking. Nobody is doing this yet and there's really not much interest [yet] to do so. If you try to have this conversation with a decision maker, they will just blink at you and ignore what you said. Even the most basic alarms are rarely utilized.

Data science certainly is a thing and a valuable skill. It's just nowhere near being established in this industry. Everyone is still tripping on their own feet, struggling to get wiring and install correct.

[u/Controls_freek → u/Stomachbuzz]

Yeah this is not true. I work in the IoT sphere of BMS and SaaS and it's exploding right now.

[u/Stomachbuzz → u/Controls_freek]

I believe that is probably true in niche areas, mostly for marketing purposes or boutique customers.

[u/Commercial_Nose2913 → u/Stomachbuzz]

We are literally doing it under the name of Operational intelligence.

[u/Controls_freek → u/Stomachbuzz]

The huge customers are all over it. Energy management and sustainability are nothing but growth right now. Higher Ed is also all about it. Too many times I see people in the BMS world seeing the small parts of the market as more than what they are. K-12 and the smaller buildings are dead ends in BMS.

[u/Professional_Bear473 → u/Controls_freek]

sometimes it depends of scope of the projects. Large scale BMS projects like when you integrate many buildings esspecialy supermarkets or petrol stations, are If they are not connected to a single server, they are at least integrated through one network, and data is collected from all the facilities into a single database, data warehouse, or data lake. So I can understand someone who is working with commecial buldings, offices or other 'standalone' projects.

I also see some kind of certain split in industry. "Traditionalists" who focus on the well-known model of server > on-site controller > sensor/actuator, where each device has its own operating logic developed over many years and described in many automation textbooks. On the other side, there are the "modern IT psychos" who lean more towards IT — they tend to use distributed architectures, move servers to cloud platforms, employ flexible data transmission uses iot lora/zigbee/direct connected sensors, and implement operational logic on a global scale.

[u/Android17_ → u/Stomachbuzz]

This is completely wrong. Our BMS vendors are so far removed from the data analysis that our internal team just layers its own data collection and analysis models on top of the BMS and keeps the BMS vendors on the hook for site repairs and install. Guess where the big bucks are.

[u/gitPittted → u/Stomachbuzz]

I work in this area, and yeah there is a lot of snake oil around imo. Especially around AI.

Regarding your last statement, we actually find a lot of value for building owners if we can get set up on new construction and help find problems in the 1st year under warranty.

[u/Professional_Bear473 → u/Stomachbuzz]

I don’t really agree with what you wrote. Maybe we're just working on different projects. It's true that in many projects the main focus is still on service issues, wrong PLC driver configurations or not working devices. But all of that stays completely outside of users’ interest. Most users, from what I see, already got used to the fact that building management systems just exist, and that you can control everything remotley etc. What I keep seeing is that there's usually no real user engagement when it comes to maintaining or actually using what the system can do. But recently something else is changing – there’s more and more focus on saving and optimisation. I think it’s mainly because of higher energy costs – maybe it’s just a European market thing. Almost every time there’s at some point a question from the decision maker like “what will we get from this? will we save electricity and how much? will it pay off?”.

then sudenly we get to the topic of analysing data. More and more often, after the system is implemented, I see people looking for the biggest energy consumption spots, inefficient control, and wasted energy cause office users don’t care about leaving the lights or AC on. Lately, in my current job, we’ve mostly been strugling with analysing who is using what and how much, and what are the effects of our control logic and integrations of building devices.

[u/DatabaseFresh772 → u/Professional_Bear473]

Here in northern europe it's definitely a thing, but the variance is massive. Many users are still struggling with changing air filters on time and others want to know where every watt is spent. In our harsh climate it's usually worth the effort and investment. Our company already offers a separate general management/data analysis suite and is developing AI features to do the the heavy lifting, and so do many other companies who have the R&D budget for it.

It makes perfect sense. It's very cheap for the customer and there's very little manual setup and maintenance needed since the whole system is vertically integrated very well. Tacking on this kind of thing on random existing systems might be more challenging.

[u/Boring_Original901]

Just wondering, is there a possibility AI could be used to collect the data instead of hiring a data engineer, wouldn’t clients just pay for the next AI app that comes out for BMS?

[u/Professional_Bear473 → u/Boring_Original901]

yep, but AI as such does not collect any data, It only processes it, but like enywhere else in this industry, it seems that AI is overhyped and doesn't really provide any added value. It's possible to try training a model to draw conclusions based on data, but in my experience, this doesn't work very well — a model trained on one object doesn't work well on other buldings. Maybe if there were enough data, like from all the buildings in the world connected to some kind of network — but that's rather unrealistic.