balancing. We store data in PostgreSQL databases or DynamoDB tables depending on use case, and utilize Apache Spark for large... point of sale application is an Android application written in Java and Kotlin. Apache Pulsar and Camel are used extensively...
and operates in 150 locations across 46 countries. Qualifications: Skills Required: Digital : Apache Spark~Digital : Databricks..., with expertise in ETLELT pipelines using Databricks and Apache Spark. Proven experience in data engineering performance tuning...
, Apache Spark, Elastic Search Experience with data management through vendor feeds. WHAT YOU CAN EXPECT FROM MORGAN...
). We integrate with various data sources, including Apache Hive, S3, HBase, and more, using Python, Spark, or R. We're... with data science and machine learning tools (R, Python, Tensorflow, Spark, mlflow etc.). Experience with AI/ML orchestration...
architecture leveraging technologies such as Apache Iceberg, AWS Glue, Redshift, and Atlan for governance. This position requires..., Informatica) to modern AWS-native services. Implement and manage a data lake architecture using Apache Iceberg and AWS Glue...
- Apache Spark, Kafka, Flink, Pinot, Trino Experience in building and maintaining large distributed systems at scale. Solid...
into centralized data platforms. Azure Databricks & Spark: Build and optimize big data processing workflows using Apache Spark... of Spark, Delta Lake, and distributed data processing. Data Visualization: Familiarity with Power BI and data modeling...
with Hadoop ecosystem (HDFS, Hive, Spark, YARN). Proficiency in Apache Spark (PySpark/Scala) for big data processing. Hands..., and maintain Big Data solutions using Hadoop ecosystem tools (HDFS, Hive, Spark, Sqoop, Oozie, etc.). Implement and optimize data...
with cross-functional teams Azure DevOps Apache Spark, Python SQL proficiency Azure Databricks knowledge Big data... Lake, Azure Blob Storage, Azure Data Factory, Apache Spark, Python, SQL, Azure Databricks, Big Data technologies Desired...
architecture leveraging technologies such as Apache Iceberg, AWS Glue, Redshift, and Atlan for governance. This position requires..., Informatica) to modern AWS-native services. Implement and manage a data lake architecture using Apache Iceberg and AWS Glue...
: Proficiency with Python and SQL; hands-on experience with Snowflake, AWS (e.g., S3, Lambda), Amazon Redshift, Apache Spark... services; implement workflow orchestration with Apache Airflow and similar tools; manage environments for reliability...
, RESTful micro services, Ignite, In-Memory Computing, Spark, PostgreSQL, Cassandra, Kafka, Yellowbrick (Apache Ignite...
governance; ideally those relevant to transaction monitoring and sanctions screening Proficiency in Databricks and Apache Spark...: Skilled in developing scalable data pipelines and performing complex data analysis using Databricks and Apache Spark...
, Ray, Vertex AI etc. Experience building data platform product(s) or features with (one of) Apache Spark, Flink...
and familiarity with CI/CD. Hands-on with some of: Kafka/Flink, Spark, Delta/Iceberg, Kubernetes, NoSQL/columnar stores. Proven... & Orchestration Tools: Apache Airflow: The industry standard for scheduling, automating, and monitoring data pipelines...
and decision-making. Develop and optimise ETL processes using Databricks and Apache Spark Design and implement efficient data... in data engineering or a related field Strong proficiency in Databricks platform and Apache Spark Excellent SQL skills...
Required: Strong expertise in SQL Apache Spark & Scala development Big Data technologies Basic Qualifications - 3+ years of data...
, Ray, Vertex AI etc. Experience building data platform product(s) or features with (one of) Apache Spark, Flink...
, or Pulsar for real-time event ingestion and processing Knowledge of Apache Flink, Spark Streaming, or Kafka Streams Role... Frameworks: Knowledge of Apache Flink, Spark Streaming, or Kafka Streams. Exactly-Once Semantics: Ensuring accurate...
/ Spark (Big Data) Elasticsearch (Search Engine) Apache Kafka (Stream Processing) TensorFlow / PyTorch (Machine Learning...#), Angular Microservices & Architecture: Spring Cloud, Docker, Kubernetes, Istio, Apache Kafka, RabbitMQ, Consul, GraphQL...