Python or other programming languages utilized for data manipulation. Experience with ETL tools such as Apache Airflow... providers including Apache PySpark AWS Glue GCP DataProc/DataFlow and Azure ADF/ADLF. Understanding of data warehousing...
solutions using Apache Spark Creates secure and high-quality production code and maintains algorithms that run synchronously..., Messaging (MQ & Kafka), application performance testing and tuning, Agile. Experience in Spark, Scala, Python writing reusable...
: PostgreSQL, MySQL, Snowflake, BigQuery, Redshift. Expert-level experience with data processing frameworks such as Apache Spark..., PySpark, and distributed computing concepts. Hands-on experience with ETL/ELT frameworks: Apache Airflow, dbt, Fivetran...
. Toast stores data in a set of sharded Postgres databases & DynamoDb, and utilizes Apache Spark for large scale data... a microservice architecture written using Java and DropWizard, with services communicating in event-driven fashion using Apache...
. Demonstrable ability to write & optimize SQL and spark jobs. Some experience in Apache Kafka and Airflow is a prerequisite as well... analysis libraries (like Spark GraphX, Python NetworkX etc.) is a plus. · · A Palantir Foundry Certification (Solution...
, and using ML Engineering platforms like Sagemaker, MLFlow, Kubeflow, or other platforms Big data - Hive, Spark, Hadoop, queuing... system like Apache Kafka/Rabbit MQ/AWS Kinesis Ability to quickly adapt to new technology and be innovative in creating...
, and maintain scalable data pipelines and ETL processes using Databricks and Apache Spark. Collaborate with data engineers..., and best practices for future reference and team enablement. Minimum Skills Required: Strong proficiency in Databricks, Apache Spark...
and expert level production experience with Apache Spark, MLLib Experience using Azure/AWS to build end to end distributed...
Software (OSS) like Apache Airflow, Iceberg, Spark and contribute back to the OSS community. Who you are We're... operates on a wide range of tech stacks including Airflow, Spark, SQL, Kafka, Flink, Hive MetaStore, Trino, Pinot, Python, Java...
knowledge and/or experience of Distributed Computing frameworks such as Apache Spark, Apache Flink, Hadoop, DASK or Ray..., Mongo DB / Non-SQL and SQL database management Apache Spark / Flink / Hadoop / DASK / Ray Azure / GCP, Kubernetes, CI/CD...
Apache Spark Framework. Monitor, manage, validate, and synthetic test data extraction, movement, transformation, loading... of experience in the Apache Spark/DataBricks framework (Python/Scala) Databricks and AWS developer/architect certifications...
Apache Spark Framework. Monitor, manage, validate, and synthetic test data extraction, movement, transformation, loading... of experience in the Apache Spark/DataBricks framework (Python/Scala) Databricks and AWS developer/architect certifications...
or data lakehouse architecture. · Experience integrating GCP data with Looker / Tableau / Power BI. · Knowledge of Apache Beam..., Spark, or Airflow DAGs. · GCP Certification (e.g., Professional Data Engineer) is a strong plus. Years of experience...
. Required Technical Expertise: Programming & Scripting Languages: Advanced proficiency in Python, R, Spark, SQL, PL/SQL, and UNIX Shell... Extensive experience with Informatica PowerCenter, Informatica Intelligent Cloud Services (IICS), StreamSets, and Apache Kafka...
pipeline orchestration tools like Azure Data Factory, Amazon data Pipeline, Apache Spark, Databricks · Hands-on experience... with one or more of stream & batch processing systems: Kafka (Confluent cloud, open source), Apache Storm, Spark-Streaming, Apache Flink, Kappa...
. Experience with SQL databases such as Sybase and Oracle, NoSQL databases such as HBase, Cassandra, and tools such as Apache Spark...
experience working with data integration teams. 1+ years of in-depth experience developing data pipelines within an Apache Spark...
experience working with data integration teams. 1+ years of in-depth experience developing data pipelines within an Apache Spark...
experience working with data integration teams. 3+ years of in-depth experience developing data pipelines within an Apache Spark...
experience working with data integration teams. 3+ years of in-depth experience developing data pipelines within an Apache Spark...