apache

Recursos de programación de apache
Going from imperative code to a reactive programming model enables us to scale our apps in ways that aren't possible with a scale-out approach. In this session, the presenter discusses & demonstrates: * How Project Reactor builds on reactive streams to help you create performant & scalable reactive microservices * Message brokers & streaming platforms like RabbitMQ & Apache Kafka * How Spring Cloud Stream leverages Reactor to provide fully reactive pipelines for system-wide (ridiculous!) scalability The presenter will code all examples using 100% open source software live and in real time. Speaker: Mark Heckler, Developer Advocate, Pivotal Software, Inc. Mark Heckler is a Java Champion, published author, conference speaker, and Spring Developer & Advocate for Pivotal developing innovative production-ready software at velocity for the Cloud and IoT applications. He has worked with key players in the manufacturing, retail, medical, scientific, telecom, & financial industries and various public sector organizations to develop & deliver critical capabilities on time and on budget. Mark is an OSS contributor and author/curator of a developer-focused blog (https://www.thehecklers.com) & an occasionally interesting Twitter account (@mkheck).
These are the best podcast/talks/ebooks I've seen/listen to recently.PodcastsScreaming in the cloud: How to Grade DevOps Teams with Nicole Forsgren, PhD Software Engineering Daily Facebook Engineering Process with Kent BeckSoftware Engineering Daily Facebook Parse Acquisition (Part 1) with Charity Majorshttps://www.infoq.com/podcasts/high-performance-cultures/ Randy Shoup on Creating High-Performance Cultures InfoQ Culture podcastDevops / Agile cultureBetter value sooner...
These are the best podcast/talks/ebooks I've seen/listen to recently.PodcastsScreaming in the cloud: How to Grade DevOps Teams with Nicole Forsgren, PhD Software Engineering Daily Facebook Engineering Process with Kent BeckSoftware Engineering Daily Facebook Parse Acquisition (Part 1) with Charity Majorshttps://www.infoq.com/podcasts/high-performance-cultures/ Randy Shoup on Creating High-Performance Cultures InfoQ Culture podcastDevops / Agile cultureBetter value sooner...
Software Developer at Streamlio Ivan is a software developer for Streamlio, where he works on Apache Pulsar and Apache BookKeeper. He's been involved with BookKeeper since it’s early days in Yahoo Labs Barcelona and also worked on the predecessor systems to Pulsar at Yahoo. His expertize is in replicated logging, distributed systems, and networking though often not at the same time.
Este taller se divide en dos partes, una teórica y otra práctica. Primero se explicará qué es Kubernetes, dónde se utiliza, para qué y su funcionamiento interno en detalle. Cuando los conceptos estén claros se hará una práctica guiada, cada uno en su ordenador, donde probaremos una pequeña demo con Minikube para trastear un poco. Es necesario instalar: VirtualBox (https://www.virtualbox.org/wiki/Downloads) Minikube (https://kubernetes.io/docs/tasks/tools/install-minikube/) JMeter (https://jmeter.apache.org/download_jmeter.cgi) opcional, pero recomendable para hacer pruebas finales -------------------- Todos nuestras charlas In-House en: https://www.youtube.com/playlist?index=1&playnext=1&list=PLKxa4AIfm4pVVBeMkXMz2BkPo9_Z3KJxk ¡Conoce Autentia! Twitter: https://goo.gl/MU5pUQ Instagram: https://lk.autentia.com/instagram LinkedIn: https://goo.gl/2On7Fj/ Facebook: https://goo.gl/o8HrWX
Big Data examples always give the correct answers. However, in the real world, Big Data might be corrupt, contradictory or consist of so many small files it becomes extremely hard to keep track - let alone scale. A solid architecture will help to overcome many of the difficulties. Floris will talk about a real-world implementation of a massively scalable ETL architecture. Two years ago, at the time of the implementation, Airflow just became part of Apache and still left many features to be desired for. However, requirements from the start were thousands of ETL tasks per day on average, but on occasion, this could become hundreds of thousands. The script-based method that was in place was already not capable to meet the requirements on a day to day basis and needed to be replaced as soon as possible. So this custom framework was rolled out in just 8 weeks of development time.
The talk is about how Apache Pulsar can have topic backlogs of unlimited size, opening up a whole array of Big Data use-cases that are not possible with other messaging systems. We also delve into tiered storage, which can make these massive backlogs very cheap. Messaging systems are an essential part of any real-time analytics engine. A common pattern is to feed a user event stream into a processing engine, show the result to the user, capture feedback from the user, push the feedback back into the event stream, and so on. The quality of the result shown to the user is often a function of the amount of data in the event stream, so the more your event stream scales, the better you can serve your users. Messaging systems have recently started to push into the field of long-term data storage and event stores, where you cannot compromise on retention. If data is written to the system, it must stay there. Infinite retention can be challenging for a messaging system. As data grows for a single topic, you need to start storing different parts of the backlog on different sets of machines without losing consistency. In this talk, I will describe how Pulsar uses Apache BookKeeper in its segment oriented architecture. BookKeeper provides a unit of consensus called a ledger. Pulsar strings together a number of BookKeeper ledgers to build the complete topic backlog. Each ledger in the topic backlog is independent of all previous ledgers with regards to location. This allows us to scale the size of the topic backlog simply by adding more machines. When the storage node is added to a Pulsar cluster, the brokers will detect it, and gradually start writing new data to the new node. There’s no disruptive rebalancing operation necessary. Of course, adding more machines will eventually get very expensive. This is where tiered storage comes in. With tiered storage, parts of the topic backlog can be moved to cheaper storage such as Amazon S3 or Google Cloud Storage. I will also discuss the architecture of tiered storage, and how it is a natural continuation of Pulsar’s segment oriented architecture. Finally, if you start storing data for a long time in Pulsar, you may want a means to query it. I will introduce our SQL implementation, based on the Presto query engine, which allows users to easily query topic backlog data, without having to read the whole thing.
El desarrollo de aplicaciones sobre Big Data está evolucionando de un modo muy rápido, tanto los frameworks y las plataformas, como las arquitecturas que podemos crear con ellos. Apache Flink es una plataforma de código abierto preparada para el procesamiento de datos tanto en streaming como en batch. En esta charla veremos cuáles son los componentes principales de Flink, cómo comenzar un proyecto desde cero, las herramientas disponibles y cómo no, algo de código Java. También veremos algunos casos de uso reales de cómo estamos utilizando Flink en el área de I+D de la división de Telecomunicaciones, Medios y Servicios de Altran, tanto para realizar análisis en real-time como en batch.
Uno de los principales casos de filtración de datos tuvo como protagonista al software tanto en su origen como el análisis. Los Panama Papers son los protagonistas del episodio de hoy. Sigue adelante el curso de Leaflet en la Zona Premium donde estamos aprendiendo a crear mapas interactivos Open Source, una alternativa real a Google Maps. Además, cuarto episodio del audiocurso sobre marketing para programadores. El centro de todo, el código. Continuo además avanzando en el reto de los 100 dí...
Por Miguel Ángel Fernández Díaz Apache Spark se ha convertido en el framework de analítica referencia dentro del mundo del Big Data. Uno de sus módulos más populares es Spark SQL, que actúa como nexo de unión entre el propio mundo Big Data y el mundo de Business Intelligence. Sin embargo, ¿a qué retos se enfrenta, por ejemplo, un consultor de BI cuando va a empezar a utilizar Spark SQL?, ¿tiene Spark SQL un lenguaje SQL estándar?, ¿qué diferencias hay respecto a una base de datos relacional? ... Más información: https://t3chfest.uc3m.es/2019/programa/sparksql-de-basico-a-optimizacion ------------------- T3chFest es un evento sobre nuevas tecnologías e innovación anual gratuito celebrado en Madrid. Consulta en nuestro canal las charlas de las pasadas ediciones. Sitio web: https://t3chfest.uc3m.es Twitter: https://twitter.com/t3chfest