- Full Time
🌎 RemotePosted on: 11 September, 2024
Big Data Distributed Systems Engineer (Remote)
100% remote job
Big Data Distributed Systems Engineer | Insikt AI | Spain
We are Insikt Intelligence and our passion is building easy-to-use Artificial Intelligence tools for helping companies and public institutions vital intelligence from digital sources data by applying Natural Language Processing and Machine Learning techniques to live data streams.
Insikt’s mission is to harness our technology for the good of society by putting tools with the potential to prevent crime and harmful content. Recently, Insikt has been acquired by Logically, a UK company that shares the same mission and purpose.
Position Overview:
We are seeking a talented and experienced Big Data / Distributed Systems Engineer to join our dynamic team. In this role, you will be responsible for designing, implementing, and optimizing large-scale data processing pipelines using Google Cloud Platform (GCP), Apache Spark, and GraphFrames. Your work will be crucial in enabling our data-driven initiatives, with a particular focus on analyzing and understanding social networks.
Key Responsibilities:
– Design and Development: Architect, develop, and maintain scalable and reliable data processing pipelines using GCP, Apache Spark, and GraphFrames, with a focus on social network data.
– Data Integration: Collaborate with data scientists, analysts, and other stakeholders to integrate diverse data sources into our big data platform, including data from social networks.
– Optimization: Optimize data processing jobs for performance, cost-efficiency, and reliability on distributed systems.
– Social Network Analysis: Apply techniques for analyzing social network structures, community detection, and network dynamics.
– Monitoring and Troubleshooting: Implement monitoring solutions and troubleshoot complex distributed systems issues to ensure the smooth operation of data pipelines.
– Collaboration: Work closely with cross-functional teams to understand data requirements, provide technical insights, and contribute to the overall architecture of the data platform.
Key Skills and Qualifications:
Preferred Qualifications:
Show more
Show less
Tagged as: remote, remote job, virtual, Virtual Job, virtual position, Work at Home, work from home
Full Time -
Full Time - 🌎 Remote
Full Time - 🌎 Remote
Full Time -