DevOps AWS Big Data (Java)

DevOps AWS Big Data (Java)

Project
World’s Most Comprehensive Cross-Device Identity big data on Businesses, Consumers and Geo Audiences. When the number of records counts hundreds of billions and datasets are terabytes on dozens of instances, every optimization counts — either performance, or cost, or disk space, or stability & backups — everything is critical.

Required

  • perfect knowledge of Linux based OS and network protocols (iptables, ssh, HTTP, FTP, mail, LDAP, Samba, etc);
  • perfect knowledge of AWS products (Amazon ES, EC2, VPC, VPN/ssh tunnel, ELB, S3, EMR, IAM, and Security) and infrastructure;
  • experience with Docker;
  • any monitoring tools (DataDog, Nagios, CloudWatch, Zabbix, Ganglia, etc.);
  • ability to write bash scripts and automate infrastructure routine actions, writing from scratch and modifying config files;
  • intermediate+ English (at least written);
  • team player;
  • a desire and ability of self-growth and self-education.


Desired

  • knowledge of any programming language, preferably Java;
  • experience with Elasticsearch, Kibana, and Logstash;
  • experience with big data projects;
  • knowledge of any Infrastructure as a Code tools (AWS Cloud Formation, Terraform, etc.);
  • experience with log analysis tools (Splunk, Papertrail, etc.);
  • good understanding of managed and not managed NoSQL databases and key-value storages;
  • knowledge of Jenkins or any other CI/CD tools;
  • pro-active approach of work in a mindset;
  • process oriented;
  • attention to details and accuracy;
  • knowledge of Azure and MS Server, IIS, MS SQL Server;
  • experience of the Hadoop related stack (HDFS, YARN, HBase, Spark);
  • experience with Ansible;
  • hands-on all AWS Re-Invent updates;
  • ability to work overtime with the same performance.


Responsibilities

  • set up and configure EC2 Linux machines and related packages (Docker, Java, PHP, node.js, Databases, etc.);
  • set up a virtual private network and secure access in AWS;
  • set up monitoring and resolve incidents (out of space, out of memory, certificates expiration, etc.);
  • set up and manage products AWS (Amazon ES, EC2, ELB, S3, EMR, etc.);
  • write bash scripts and automate infrastructure routine actions;
  • use (where possible) Infrastructure as a Code tools (AWS Cloud Formation, Terraform, etc.);
  • set up and maintain CI/CD tools (Jenkins, etc).
  • always think how to solve the task in the most cost-efficient way;
  • propose actions to reduce infrastructure cost for existing accounts and implement them once it’s approved;
  • follow all new AWS product announcements and suggest improvements to projects’ infrastructures;
  • study AWS products and related technologies (monitoring tools, log analysis tools, databases, etc) to be able to help the team with configuration and improvements;
  • monitor and keep tracking on abnormal cost spikes to answer the question of why it happened and what could be done to avoid that in future.
  • on projects with a dedicated DevOps position (full-time or part-time) timely communicate with the customer in emails in English;
  • in case of project infrastructure incident (going on or just resolved) participate in conf calls with customer biz or tech team;
  • send planned maintenance email notifications;
  • send emails on critical monitoring incidents’ fix status (sometimes with the investigated possible reason).

Quick Talk

Skype invitation, phone call or email — and the first step in establishing a long-lasting relationship is made!

+38(050)4000-575 +1(816)820-4793

welcome@videal.net

videalcorp

Quick Message

Start a conversation with us and receive expert consultancy!

×