Wagdy El-Sherbiny
Verified Expert in Engineering
Data and Database Developer
Wagdy is a senior data and database engineer with over eight years of experience with data-related and real-time mega projects. Highly proficient in SQL, databases, and data warehouses, including designing and implementing, 他管理过软件项目,并为重要客户工作过, such as ADNOC, Emarat, KNPC, and KOC. Wagdy is an expert in data pipelines, data analysis, and Python and focuses on understanding clients' needs and designing and developing state-of-art solutions.
Portfolio
Experience
Availability
Preferred Environment
Data Engineering, Databases, Data Warehouse Design, Python, Data Analysis, NoSQL, SQL, PostgreSQL, MongoDB, ETL Tools
The most amazing...
...我开发的是ADNOC在阿联酋使用的自助支付和操作站, the world's 12th largest oil company by production.
Work Experience
Python and PL/SQL Developer
Susquehanna Technology Management, Inc. (STMI) - Main
- 实现了一个Python程序来替换遗留系统, drastically reducing processing time for calculating trading volumes per symbol and exchange messages. 将处理时间从7到10小时减少到大约20到30分钟.
- 设计了一个Python脚本从url中提取数据, parse JSON information, 并将其无缝存储在SQL server数据库中,以实现高效的数据管理.
- 创建了一个shell脚本,用于从FIDO交易系统中检索数据. 实现过滤,提取重要信息,保存到CSV文件. 利用外部表方便地将数据加载到Oracle表中.
- 将最初在Python 2中编写的所有遗留Python脚本迁移到最新的Python 3.7版本,确保当前开发标准的兼容性和优化.
- 使用Jira管理bug跟踪和解决过程, ensuring seamless production support and maintenance.
- 编写了一个健壮的数据库日志记录和调试实用程序,以简化故障排除. Also, designed a Python module for easy integration into scripts, bolstering debugging capabilities.
- Revamped and reimplemented outdated systems prone to failure when processing large files with efficient and swift Python programs.
Python and PL/SQL Developer
SIG - Main
- Developed a regression testing program that validates the outcomes of one of the leading trading solutions at SIG. 该应用程序便于高层管理人员验证数据结果.
- Built a real-time data pipeline that monitors three trading symbols' order trades and extracts data as of the ACT messages protocol. 该脚本提取和解析数据,然后将其加载到Oracle数据库中.
- Participated in PySpark script development, converting legacy reports into new ones that optimized execution time and processed the vast amount of daily data, 500+ million transactions.
- 开发了一个ETL,用于解析XML文件将数据加载到Oracle数据库表中.
- 使用SSRS创建了许多报告,并使用qliksense创建了一个现代可视化仪表板.
- Converted a legacy Perl program into Python. The scripts aim to process trading orders and calculate the five-minute sliding average for each symbol.
Senior Data Engineer
Diyar United Company
- Collaborated with the project management team to analyze and consolidate request requirements to improve the decision-planning process and meet project deadlines.
- 领导中型项目从范围定义到执行和实施.
- 与跨职能团队合作解决问题, identify appropriate data sources, extract data, 并开发集成信息传递解决方案.
- Designed, implemented, 维护各种数据库管理系统(dbms), such as Oracle, Microsoft SQL Server, Microsoft Azure SQL, and PostgreSQL.
- Analyzed and compiled an itemized list of components needed for data migration or integration with various systems, such as Oracle Enterprise Resource Planning (ERP), SAP, and Comarch Loyalty Management (CLM).
- 参与设计和实现一个数据仓库, data marts for analysis, data mining projects, and implementing changes to existing BI systems.
- 使用SSIS工具从各种资源中收集数据, 遵循数据整理周期,存储和交付清理后的数据以供数据分析, 并开发了一个定制的管道,用于从外部系统获取和加载数据.
- Built a fast and reliable invoicing system that efficiently processes millions of transactions and integrates with the Oracle ERP or SAP system. 发票生成时间从6小时减少到20分钟.
- Identified the key reports and dashboards to satisfy the client's requirements and developed different kinds of reports like parameterized reports, ad hoc reports, 并使用SSRS和Power BI对报告进行深入研究.
- Troubleshooted data issues, analyzed and corrected error and performance logs, created backup routines, and prepared system deployments and upgrades. 培训各个业务单元团队有效地使用流程、工具和资源.
Full-stack Developer
Management Development Center (MDC)
- 参与了Rhazes解决方案的开发,包括所有的医疗, clinical, and administrative modules needed to manage and operate a hospital in a smooth and user-friendly way.
- Developed SCHOOLIZER software used to efficiently manage all the systems and processes related to running any educational institute.
- 排除故障并修复与称为MUST的电子学习系统相关的生产问题.
Teaching Assistant
Misr University for Science and Technology
- Managed, planned, and prepared the learning materials and courses related to computer science for different classes and levels.
- Used different educational methods to promote a positive learning environment and organized group work projects and brainstorming sessions.
- 为不同的评估设定要求,并记录学生的发展情况.
- 与计算机科学领域的变化和发展保持同步.
Experience
ADNOC Smart Wallet
http://smart.adnocdistribution.ae/I was the software engineer who participated in and developed all objects related to the data area, including databases such as triggers, stored procedures, jobs, data warehouse, pipelines, and reporting. I created a set of systems, including the following:
• A card management system (CMS).
• A comprehensive centralized software solution providing our clients with a user-friendly interface that integrates tightly with other back-end and external systems, such as customer management, beneficiary management, token management, transaction management, system settings, BI reports and dashboards, differential pricing, notification management, ERP integration, loyalty system, e-vouchers, coupons, data migration, 以及个性化智能卡的卡片打印系统, vehicle depot system, and invoicing.
Emarat Petroleum Company Station Management Service
http://www.emarat.ae/businesses/fleet-payment-solutions• Implementing and customizing a turnkey solution to meet the company's station management and retail services needs.
•参与系统需求收集和分析阶段.
• Analyzing and compiling components needed for data migration and integration with various systems, such as Oracle ERP, NCR, and NAMOS.
•开发数据迁移工作流,将数据迁移到使用SSIS的新系统.
•建立差异化定价体系和NCR折扣.
•使用SSRS和Power BI开发各种报告和仪表板.
Kuwait E-passport
KNPC Petrol Filling Stations
Kuwait Digital Oil Field Project
Kuwait National Guard's Fleet Management
The Ford GoBike System
http://github.com/wagdySamy/DataAnalysisWeRateDogs Analysis
Education
Bachelor's Degree in Computer Science
埃及吉萨科技大学
Certifications
Spark and Python for Big Data with PySpark
Udemy
HCIA-Big Data Course
Huawei Academy
AWS Machine Learning
Udacity
Data Analysis Professional Nanodegree Program
Udacity
Data Engineer with Python
DataCamp
MongoDB Basics
MongoDB University
Data Analysis Advanced Nanodegree Program
Udacity
Career Track Data Analyst with SQL Server
DataCamp
Big Data Introduction Course
Coursera
MCPS: Microsoft Certified Professional
Microsoft
Computer Science Diploma
Mansoura University
Skills
Libraries/APIs
Pandas, Matplotlib, NumPy, Twitter API, PySpark
Tools
Microsoft Power BI, Qlik Sense, Flink, Flume, Apache ZooKeeper, SQL Server BI, Oracle ERP, Crystal Reports, Shell, Audacity
Storage
Databases, SQL Server Reporting Services (SSRS), PL/SQL, PL/SQL Developer, Oracle PL/SQL, SQL Server Integration Services (SSIS), PostgreSQL, Data Pipelines, NoSQL, MongoDB, Azure SQL, Microsoft SQL Server, HBase, Apache Hive, Database Administration (DBA), Database Modeling, SQL Server Analysis Services (SSAS)
Languages
SQL, Python, C#, C++, C#.. NET, T-SQL (Transact-SQL), Scala, Active Server Pages (ASP), HTML, CSS, Bash
Platforms
Oracle, Apache Kafka, Linux
Paradigms
ETL Implementation & 设计、敏捷、商业智能、MapReduce、ETL、用户验收测试(UAT)
Frameworks
Hadoop, Spark, ASP.NET
Other
Data Warehouse Design, PL/SQL Tuning, Data Engineering, Data Analysis, ETL Tools, ELT, Data Wrangling, Big Data, Azure Data Factory, Software Engineering, Data Structures, Operating Systems, Algorithms, Windows Communication Foundation (WCF), Artificial Intelligence (AI), Neural Networks, Streaming, Machine Learning, Data Visualization, PETEX, DAX, Data, IIS, Analysis, Teamwork, Team Management, Data & Backup Management, Data Modeling, Production, Deployment, Software Troubleshooting, Troubleshooting, Education, Communication, University Teaching, Applied Mathematics, Statistics, Windows Services, Web Services, Linear Regression, Multidimensional Expressions (MDX), Star Schema, Big Data Architecture, Data Warehousing, Data Reporting, Optimization, Workflow Automation
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring