Chimborazo

Majestuoso volcán Chimborazo

Amigos de la blogosfera, tenía muchas ganas de escribir este post. La idea original nació hace algún tiempo, en los años universitarios, tras leer el fantástico libro Travels amongst the great Andes of the Equator del Alpinista británico Edward Whymper pero por innumerables eventos  (principalmente el abandono de este blog) nunca se concretó y hoy con el consecuente giro temático del mismo y contando la inspiración necesaria, me pongo manos a la obra.

Por cuestiones de trabajo suelo volar hasta la ciudad de Quito en numerosas ocasiones del año. Es una corta travesía que suele tomar entre 45 y 60 minutos dependiendo de las condiciones climáticas. Cuando estas son benévolas, el trayecto es tranquilo y permite disfrutar una vista a los colosos que se alinean a lo largo de la Avenida de los Volcanes, pero esto no es así siempre, siendo los Andes unos gigantes celosos, se suelen envolver en un manto nuboso que los distancia de ojos mortales.

El vista aérea del Chimborazo. Clic para ver más grande.

WF Reactive System (2)

In previous post (available here.) we gave a try to Lagom and reactive microservices. We started by creating a sample Weather forecast App with three reactive microservices. Starting by the preparation and setup of tools needed, and the creation of service API and its implementation for the producer service that will deal with request that User will generate from a future UI (We’ll have the chance of play with Play framework).

In this  part, we’ll create a consumer API and its implementation.

Part IV: Consumer API

1. Open the project in the IDE and create a copy of the producer module and rename it as wf-consumer-api.
2. Add this module to build.sbt.
3. Go to model and refactor (rename):
   1. WFMessage to WeatherForecasting.
4. Modify the Service API:
   1. Remove topic
   2. Change descriptor name to “wfconsumer”.
   3. Define functions to:
      1. Get Top Ten Weather Forecasts.
      2. Get one (the first) Weather Forecast. (findOneWFData)
      3. Get the latest Weather Forecast. (latestWF)
   4. Define the rest endpoints URI in descriptor to our consumer.
      1. Get for (findTopTenWFData)
      2. Get for (latestWF)
      3. Get for (latestWF)
   5. Remove the withTopics and withAutoAcl options of descriptor.

Part V: Implementing the Consumer Services

1. Create model package and create the case class WeatherForecasting.
2. Create companion object for Json serialization format.
3. Modify the consumer’s WFCommand
   1. Extend ReplyType[T] (previously [Done])
   2. Create SaveNewWF case class and its companion object.
   3. Delete WF case class and its companion object.
   4. Rename WFSerializerRegistry to WFConsumerSerializerRegistry
   5. Add class to WFSerializerRegistry.
   6. EventTag does not require modification.
   7. Modify WFEvent. Refactor WFChanged to WFSaved and change its signature to receive a WeatherForecast object as parameter.
   8. Remove onReadOnlyCommand
   9. In onEvent change case and add message for detail of message.
   10. Modify WFEntity to implement saving messages to handle onCommand and on Event operations for all WFEvents. It is very important to note that this Entity manages components of CQRS/ES architecture.
   11. Modify WFLoader.
       1. Rename it to WFConsumerLoader
       2. Override load and loadDEvMode.
       3. Change abstract class WFConsumerApplication
       4. Add lazy vals for binding WFService client wfService and message repository (WFRepository).
       5. Override jsonSerializerRegistry.
       6. Add persistentEntityRegistry
       7. Add readSide.register(wire[WFEventProcessor])
   12. Implement WFRepository.
       1. Create package repositories.
       2. Create private [impl] class WFRepository its constructor receives CassandraSession.
       3. Implement method for data fetching according to rest endpoints:
          1. fetchWFData: to retrieve 10 elements
          2. fetchOneWFData: to retrieve one (first) register.
   13. Implement Consumer Service
       1. Create class WFConsumerServiceImpl.
       2. Override findTopTenWFData and findOneWFData
   14. change Loader in resources/application.conf
   15. Add binds to build.sbt.

In the next post, we’ll create a simple frontend for our Reactive System.

WF Reactive System (1)

Prerequisites:

• Java 8
• Scala
• sbt
• IntelliJ IDEA

Part I: Preparation from Template

Create new lagom project from template.

sbt new lagom/lagom-scala.g8

1. Import project into IDE.
2. Refactor project-name-API to create producer API and IMPL.
   1. rename folder wfproducer-API.
   2. Rename module wfproducer-API.
   3. Change new names in build.sbt lazy vals and bindings. Note: at this point you can remove lazy vals of stream services API and its implementation created by lagom template.
3. Test by runAll. If you’ve done changes correct it’ll run.
   1. Go to: http://localhost:9000/api/hello/Martin

Part II: Define a Producer Service API

1. Delete stream-api and stream-impl services. Do not forget to remove their references (lazy val) from build.sbt
2. Since we’re building a reactive Weather forecast system we’ll handle user events on wf-producer service.
3. Rename wfreactiveService Scala class of wf-producer-API service. Take into consideration that a service API is an interface that in scala is declared with a trait, so make sure to rename using IDE’s refactor capabilities in order to keep consistency and find usages on other scala files.
4. Extract model from service api implementation.
   1. Create model package inside api.
   2. Move class to the new model package.
   3. Rename class and its companion object to WFMessage.
5. Open WFService.
   1. Change the topic to wfTemperature.
   2. Change signature of wfTemperature.
   3. Remove other functions definition.
   4. Rename greetingsTopic function to wfTopic and its signature.
   5. Include pathCall to wfTemperature in descriptor.
   6. Remove the addProperty Kafka part.
   7. Delete case class GreetingMessageChanged part.

Part III: Define a Producer Service Implementation (impl)

1. Producer commands Creation:
   1. Sealed trait WFCommand
   2. UseWFMessage
   3. WF
2. Create companion objects for commands
3. Create Serialized registry. Add all command case classes.
4. Create system events:
   1. (sealed trait) WFEvent
   2. WFChanged
5. Create Event Tag (WFEventTag) for Service.
6. Add system events case classes to Serialized registry.
7. Create WFState
8. Add WFState to Serialized registry.
9. Modify WfreactiveEntity.
   1. Rename it to WFEntity.
   2. Change Initial State
10. Implement WFLoader
    1. Rename it to WFLoader
    2. override lazy val jsonSerializerRegistry = WFSerializerRegistry
11. Implement WFServiceImpl
12. Change in application.conf values of Loader

play.application.loader = com.example.wfreactive.impl.WFLoader

(This is a guide, a draft, will be completed with images soon)

On Reactive Manifesto

Full view at Coggle

Alexander Von Humboldt

Sunset at Kazan

R Cheatsheet: Reading XLSX files

#Use the xlsx library

#By default it is not available in base R so we must install package

install.packages("xlsx")

library(xlsx)

#If we use the read.xlsx with only the filename parameter it will cause an error, because we must provide the sheetindex and point out wheter the sheet contains headers for each column

readexcelfile<-function(){
 library(xlsx)
 localcopy<-"./data/cameras.xlsx"
 cameraXlsx<-read.xlsx(localcopy)
 head(cameraXlsx)
}

#Correct use of read.xlsx

readexcelfile<-function(){
library(xlsx)
 localcopy<-"./data/cameras.xlsx"
 cameraXlsx<-read.xlsx(localcopy, sheetIndex = 1, header = TRUE)
 head(cameraXlsx)
}

R Cheatsheet: Files and Directories

Check whether a directory exists, if true then nothing happen

if(!file.exists("test")){

dir.create("test")}

 

Next Step, write a function that downloads a csv file from the internet:

downloadfileurl<-function(){

#Check if a directory exists, otherwise we create 
 if(!file.exists("data")){
 dir.create("data")}
 
 #Next step, download a file from the internet.
 
 #First we create a variable with the url which contains the data:
 fileURl<-"https://data.baltimorecity.gov/api/views/dz54-2aru/rows.csv?accessType=DOWNLOAD"
 
 #Next variable containts the location of local copy of downloaded file
 localcopy<-"./data/cameras.csv"
 
 #In order to obtain an online file we must use the download.file() function
 #Since we are working from a Windows terminal the third parameter (method) should work
 #with the default value. If you're working from a Mac, then you must specify its value to "curl"
 #because that file is available via https protocol
 download.file(fileURl,destfile = localcopy)
 
 #we check the files in that directory
 files<-list.files("./data")
 print(files)
 #Finally we print the date we downloaded that file. This is very important specially because you need
 #to be able to keep track of that file.
 datedownloaded<-date()
 print(datedownloaded)
}

After executing this function we obtain as a result:

Now we check the existence of the new file using the File Explorer: