Capture all row-based DML changes from the Oracle database and stream these changes to Kafka. The change data capture logic is based on the Oracle LogMiner solution.

  1. Test environment preparation

(1)A linux cloud server is configured as a 2 core 4G 2M bandwidth 50G system disk

(2)install docker on cloud…

In our work, we often encounter the problem of doing various aggregation calculations based on time or other dimensions.

Problem 1 : user login to a website ,statistical activity duration

Suppose there is a DataFrame of user logins to a website, for instance:

| user_name|login_date|
|smith |2020-01-04|
|bob |2020-01-04|
|bob |2020-01-06|
|john |2020-01-10|
|smith |2020-01-11|

I would like…

Eureka registry/service discovery framework

Eureka is a service discovery framework developed by Netflix, itself a REST-based service that is used to locate middle-tier services running in AWS domains for load balancing and middle-tier service failover purposes. springCloud integrates it into its subproject spring-cloud-netflix to implement the SpringCloud’s Service Discovery feature.


spark kafak flink develop

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store