2021
Goals for This Year
Goals for Next Year
- Enhance the data platform to be able to use data to make business decisions.
Projects
- Create a modern data platfrom from scratch. Learned lots of stuff along the
way.
- We'll be able to collect data and use data for innovation now.
Technology Stack
- AWS Cloud
- Amazon S3
- EKS (K8s)
- AWS Lambda
- RDS (Postgres)
- Amazon EFS
- Google Cloud
- Google BigQuery
- Google Data Studio
- Deployment
- ArgoCD
- Jenkins
- Kubernetes (K8s)
- Pulumi
- Terraform
- Data Pipeline
- Airflow
- Dagster
- Data Catalog
- CKAN
- Monitoring
- Prometheus
- Grafana
Company Building
- Help creating a data thinking workshop for the financial firm.
- Create a knowledge repo for the innovation team.
What I Learned
- Creating a modern data platform from scratch requires tons of knowledge such
as infrastructure or security. Data quality is very challenging.
- Challenges in building a data culture in an enterprise. We need an effective
communication skills, especially storytelling.
- Created data pipelines with Dagster.
- Created and managed the Kubernetes clusters. Also implemented GitOps CD using
Argo CD.
- Building a community requires lots of effort.
Outside of Work
Talks
Podcasts
Open Source