Senior Python Data & AI Engineer (Web Crawling) - Project setup requires service delivery from a nearshore country (m/f/d)

Project overview

Senior Python Data & AI Engineer (Web Crawling) - Project setup requires service delivery from a nearshore country (m/f/d)

Reference: 93643d4f-05c9-41dd-bb23-9000c64f11be

Responsibilities

• Analysis and Indexing of Appropriate Web Sources: The contractor shall identify and evaluate appropriate sources
• Design of the crawling and extraction solution:Development of a technical target architecture
• Development of Crawling, Extraction, and AI Components
• Data Structuring and Provision: Transformation of the extracted information
• Quality Assurance and Domain Validation: Implementation of procedures and a Databricks app
• Pilot Testing
• Documentation and Handover

Requirements

Main advantages:

• Web Crawling & Web Scraping: Scrapy, Beautiful Soup, or comparable frameworks, HTML, DOM, and table analysis, working with dynamic web pages and download structures
• Information Extraction & Generative AI: LLM & NLP, prompt engineering and structured output formats, classification, mapping, and duplicate detection
• Interface and Document Processing: Rest APIs, PDF and datasheet extraction, Table extraction and schema mapping
• Data Engineering & Databricks, Python, Pandas and PySpark, Delta Lake, and development of Databricks apps
• A very renowned company
• Possibilities to work remote

Project details

asap

2 MM++

• Start: ab sofort
• Duration: 2 Monate
• Remote: onsite
• Location: Dortmund, Deutschland, DE

Source notes

IT is and always has been our core business that laid the foundation for Hays' success. We are the biggest privately owned IT recruitment agency in Germany and offer the best jobs for every career level – whether you are interested in vacancies in agile SMEs or international DAX groups. Hays masters the entire IT job spectrum, from support to software architecture or digitalisation – thanks to our broad portfolio, we have something for everyone. In the last decades, we were able to support numerous IT experts with choosing the right path for a successful career, positioning ourselves as their lifelong partner.

Our highly specialised consultants can cater to your every wish and expectation and will prepare you for interviews and contract negotiations. Give it a try and learn what the market has to offer – our services are free of charge, non-binding and discreet! We look forward to hearing from you.

My contact at Hays:

My contact person:

Michelle Winkels

Referencenumber:

*

Make contact:

Email: *@*.de