Trending December 2023 # Etl Pipeline With Google Dataflow And Apache Beam # Suggested January 2024 # Top 20 Popular

You are reading the article Etl Pipeline With Google Dataflow And Apache Beam updated in December 2023 on the website We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested January 2024 Etl Pipeline With Google Dataflow And Apache Beam

This article was published as a part of the Data Science Blogathon.


Processing large amounts of raw data from various sources requires appropriate tools and solutions for effective data integration. Many companies prefer to work with serverless tools and codeless solutions to minimize costs and streamline their processes.  Building an ETL pipeline using Apache beam and running it on Google Cloud Dataflow is an example of creating an ODS solution from EPAM.

Challenge 1: A data integration solution supporting very different types of data sources

The client contacted EPAM with a challenging task – to create a data integration solution for data analysis and reporting with the ability to integrate with machine learning and AI-based applications. The data integration solution had to collect data from different types of data sources (SQL, NoSQL, REST) ​​and transform the data for reporting, discovery, and ML. The client was interested in an analytical tool like Tableau or Data Studio, looking for a solution that could easily integrate with many traditional and non-traditional data sources and be flexible to any change.

The client expected the developed solution to include a reliable and flexible data pipeline architecture and an internal common data model for abstracting incoming data extracted from various data sources.

What is an ETL pipeline: ETL vs. Data Pipeline

A data pipeline is a process of moving data from a source to a destination for storage and analysis. A data channel generally does not specify how the data is processed along the way. One of the properties of a data channel is that it can also filter data and provide fault tolerance.

If this is a data pipeline, what is an ETL pipeline? In the ETL data pipeline, data is loaded (Extract), processed (Transform), and passed to the target system (Load).

Extract-transform-load pipeline: enables data migration from the source system to the new repository; centralizes and standardizes multiple resources for a consolidated v, and provides a large dataset for BI and analytics. Broadly speaking, ETL is a sub-process, while “data pipeline” is a broader term that represents the entire data transfer process.

EPAM solution

The EPAM team identified the first step as creating an ODS (Operational Data Store); central data storage from multiple systems without specific data integration requirements. ODS makes data available for business analysis and reporting by synthesizing raw data from multiple sources into a single destination.

Unlike a data warehouse that contains static data and executes queries, ODS is an intermediary for the data warehouse. It provides a consolidated repository available to all systems that frequently rewrite and change data, creates reports at a more sophisticated level, and supports BI tools.

When looking for a self-service integration platform, the EPAM team wanted a solution for external teams and external resources that would not be as costly as P2P integration.

First, we found a platform to run our solution on. Then we decided on a solution. We finally put it all together.

What is a data stream?

The Google Cloud Platform ecosystem provides a serverless data processing service, Dataflow, for running batch and streaming data feeds. As a fully managed, f, fast, and cost-effective data processing engine used with Apache Beam, Cloud Dataflow enables users to develop and execute a variety of data processing, extract-transform-load (E, TL), and batch and streaming patterns.

Dataflow is the perfect solution for building data pipelines, tracking their execution, and transforming and analyzing data, as it fully automates operational tasks such as resource management and optimizing your pipeline performance. In Cloud Dataflow, all resources are provisioned on demand and automatically scaled to meet demand.

Dataflow works for both batch and streaming data. It can process different pieces of data in parallel and is designed for big data processing. Dataflow is the perfect solution for automatically scaling resources, balancing dynamic work, reducing the cost of processing a data record, and delivering ready-to-use real-time AI patterns. The range of features allows Dataflow to perform complex pipelines using basic and specific custom transformations.

With Dataflow providing a range of opportunities, your results are only limited by your imagination and your team’s skills.

Why does EPAM choose Dataflow?

Besides Dataflow, there are other data processing platforms. Google’s Dataproc service is also an option, but the EPAM team liked that Dataflow is serverless and automatically adds clusters when needed. Most importantly, Google intended Apache Beam programs to run on Dataflow or the user systems.

Google promotes Dataflow as one of the main components of the big data architecture on GCP. With the ability to extract data from open sources, this serverless solution is native the to Google Cloud Platform, enabling rapid implementation and integration. Dataflow can also run in ETL solution because it has: building blocks for operational data stores and data warehouses; data filtering and enrichment pipelines; PII de-identification pipeline; function for detecting anomalies in financial transactions; and export logs to external systems.

What is Apache Beam?

What is Apache Beam? Apache Beam, which evolved from several Apache projects, emerged as a programming model for creating pipelines for data processing. The Apache Beam framework does the heavy lifting for large-scale distributed data processing.

You can use Apache Beam to create pipelines and run them on cloud runners such as Dataflow, Apache Flink, Apache Nemo, Apache Samza, Apache Spark, Hazelcast Jet, and Twister2.

Why did EPAM choose Apache Beam?

First, Apache Beam is very efficient and easy to use with Java. Unlike Apache Spark, Apache Beam requires less configuration. Apache Beam is an open, vendor-agnostic, community-driven ecosystem.

Beam has four basic components:

A pipeline is a complete process consisting of steps that read, transform, and store data. It starts with an input (source or database table), includes transformations, and ends with an output (sink or database table). Transformation operations may include filtering, joining, aggregation, etc., and are applied to data to give it the meaning and form desired by the end user.

A collection is a specialized container of almost unlimited size representing a data set in a pipeline. Collections are the input and output of the real transformation operation.

PTransform is the data processing step inside your pipeline. Whatever operation you choose—data format conversion, math calculation, data grouping, combining, or filtering—you specify it, and the transformation performs it on each collection element. The P in the name stands for parallel because transformations can run parallel across many distributed workers.

Apache Beam code examples

Although Java or Python can be used, let’s look at the Apache Beam code pipeline structure using Java.

Here is an example created using Dataflow. To create and operate a pipeline, you must

Create a PCollection

Use the PTransforms sequence

Run the pipeline

Here’s an example built using the Apache Spark runner that replicates the code but uses a different Runner options fragment.

The first transformation is ‘’, and the output is a P Collection with string lines of text from the input file. The second turn is the rows of strings in the P Collection. The third transformation is empty word filtering. The fourth transformation counts the number of times a word appears. This map transformation applies a function to each element in the input PC collection and produces a single output element. Every word count counts. The fifth and final transformation formats the Map Elements into strings of Type Descriptors and produces an output text file.

This example is done with standard Beam frame transformations.

Now let’s see how to run a pipeline with user-defined custom transformations. After the channel is created, the Read transforms text file is used. The Count transformation is a custom transformation that counts words. The map transform uses the new FormatAsTextFn custom function to format each word count occurrence into a string. The strings are then passed to the output file.

Now let’s see how to create compound transformations. Here we use the ParDo steps (transformation for generic parallel processing) and the transformations in the SDK to count words. Transform subclass Count Words have two complex transformations. ParDo extracts the words, and the transformation provided by the SDK runs Count.perElement.

A similar algorithm is used for word extraction. We just split the row into words and filter out the empty values. In this exam,  plus, we enter explicit DoFns. This function gets one element in the input PCollection and the receiver in the resulting output file.

Challenge 2: unexpected context

The customer was satisfied with the solution that EPAM designed based on Dataflow. A fully serverless data processing system, built on GCP and using Google’s big data architecture, was exactly what they needed to support multiple resources and use machine learning and AI.

However, after presenting the solution, the customer revealed an unexpected piece of information: the databases were critical and sensitive. Therefore, the customer could not trust Google or use most GCP services for security reasons.

Our solution

In response, the EPAM team leveraged available software services and built a separate ETL solution for each data source. In the future, each new data source will be added by generating a new project from a common ETL archetype and applying business logic code. When designing the customer’s project pipeline, the EPAM team studied the customer’s data sources and decided to unify all data in the workspace before loading it into the data warehouse. The workspace acts as a buffer and protects data from corruption.

Essentially, we have arrived at a Lambda architecture variant that allows users to process data from multiple sources in different processing modes.

The EPAM solution is open to expansion and external integrations. The EPAM solution has two independent phases that increase system reliability and expand the possibilities for external systems by using a batch system and a stream processing system in parallel. External systems can passively integrate through data source access and actively send data to the staging area, either to Google BigQuery or directly to the BI reporting tool. A workspace in the project pipeline allows users to clean data and combine multiple data sources.

EPAM Integrators have created a software architecture design ideal for data integration and a GCP Dataflow framework that can be used as a starting point for building ETL pipelines. EPAM’s software architecture design enables the use of ready-made ETL solutions and codeless integration. Over time, the customer and integrators will increase the number of integrations and extend the Apache Beam framework with new components. This will greatly facilitate subsequent integrations.


A data pipeline is a process of moving data from a source to a destination for storage and analysis. A data channel generally does not specify how the data is processed along the way. One of the properties of a data channel is that it can also filter data and provide fault tolerance.

Our conclusions from the construction of the ETL gas pipeline for the customer can be summarized as follows:

Using a serverless approach significantly speeds up data processing software development. The Apache Beam framework does the heavy lifting for large-scale distributed data processing.

Apache Beam is a data processing pipeline programming model with a rich DSL and many customization options.

A framework-style ETL pipeline design enables users to build reusable solutions with self-service capabilities. A serverless and decoupled architecture is a cost-effective approach to meeting your customers’ needs.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.


You're reading Etl Pipeline With Google Dataflow And Apache Beam

Google Speaks Loudly With Google Talk

Google Speaks Loudly with Google Talk

Usernames on Google Talk are associated with their email accounts. While chatting with other users, the Googler has the option of pressing an embedded button in the messaging window or the application dashboard to instantly Gmail the end user. An inbox link is the premier navigational link on the Google Talk application, even if the targeted friend is offline. When the friend is online, a telephone button is also served for making VOIP calls.

The integration of Google Talk throughout GMail and vice versa is a smart play for Google, who is hurting for registered users. That’s what Google Talk will bring to the Google network, registered users. According to a ComScore MediaMetrix survey, over 81 million Americans used Instant Messaging systems in July. Over 30 million used AIM, 23 million used MSN Messenger and 23 million used Yahoo Messenger. Additionally, Skype enjoys over 51 million international users.

If Google Talk takes off to reach heights anywhere near Google Search has, Google should achieve Google Talk usage of maybe 15-20 million users over the next year. 15-20 million users who are routinely plugged into Google and GMail, using Google apps for major communications, mail, and eventually desktop search, feed aggregation (via Sidebar), Google Local and of course web search.

Until recently, the stickiest user base Google had was its Orkut social network, which took off like wildfire in Brazil but never really reached the capacity of LiveJournal or Friendster in the United States. Now, if one is not interested in having a desktop application which sorts the web’s information into an easy to monitor Google sideshow (in Sidebar), that person may be interested in using Google Talk to make computer to computer calls via Google or keep up with their friends and colleagues over Google messaging. In the span of one week, Google has introduced two strong offerings in Sidebar and Google Talk which have filled that Google void.

Like Sidebar, Google Talk is also offering plug-ins and add ons along with an open invitation to developers to put together apps compatible with Google Talk. From the Google Talk site “Google Talk, which enables users to instantly communicate with friends, family, and colleagues via voice calls and instant messaging, reflects our belief that communications should be accessible and useful as well. We’re committed to open communications standards, and want to offer Google Talk users and users of other service providers alike the flexibility to choose which clients, service providers, and platforms they use for their communication needs.”

Apache Flume: Data Collection, Aggregation & Transporting Tool

This article was published as a part of the Data Science Blogathon.


Apache Flume is a platform for aggregating, col . It features its query processing engine, allowing it to alter each fresh batch of data before sending it to its designated sink. It is designed to be flexible.

The design of built on streaming da ws, which makes it very simple and easy to use. Apache Flume has several adjustable dependabilities, recovery, and failover features that come to our aid when we need them.

Apache Flume has many features and lets us have a look at some of the notable and essential elements of Flume:

Flume efficiently ingests log data from many online sources and web servers into a centralised storage system (HDFS, HBase).

Flume is also used to ingest massive amounts of event data produced by social networking sites like Facebook and Twitter and e-commerce sites like Amazon and Flipkart and log files.

Flume can handle many data sources and destinations.

Flume can handle multi-hop flows, fan-in-fan-out flows, contextual routing, etc.

The flume may be horizontally scaled.

We can quickly pull data from many servers into Hadoop using Flume.

Benefits of using Flume

Using Flume has many benefits. Let us have a look at the benefits of using Flume:

We may store the data in any centralised storage using Apache Flume (HBase, HDFS).

Flume has a feature called contextual routing.

Flume transactions are channel-based, with each communication requiring two transactions (one sender and one recipient). It ensures that messages are delivered on time.

When the incoming data rate exceeds the rate at which it can be written to the destination, Flume works as a middleman between data producers and centralised storage, ensuring a continual flow of data between them.

Apache Flume Architecture

Flume Architecture consists of many elements; let us have a look at them:

Flume Source

Flume Channel

Flume Sink

Flume Agent

Flume Event

Flume Source

A Flume Source can be found on data producers like Facebook and Twitter. The source gathers data from the generator and sends it to the Flume Channel in the form of Flume Events. Flume supports a variety of sources, including Avro Flume Source, which connects to an Avro port and gets events from an Avro external client, and Thrift Flume Source, which connects to a Thrift port and receives events from Thrift client streams, Spooling Directory Source, and Kafka Flume Source.

Flume Channel

A channel is a transitory storage that receives events from the source and buffers them until sinks consume them. It serves as a link between the authorities and sinks. These channels are entirely transactional and can connect to an unlimited number of sources and sinks.

Flume supports the File channel as well as the Memory channel. The file channel is persistent, which means that once data is written to it, it will not be lost even if the agent restarts. Channel events are saved in memory, and therefore, it is not long-lasting but highly rapid.

Flume Sink

Data repositories such as HDFS and HBase include a Flume Sink. The Flume sink consumes events from Channel and saves them in HDFS or other destination storage. There is no need for a sink to give possibilities to Store; alternatively, we may set it up so that it can deliver events to another agent. Flume works with various sinks, including HDFS Sink, Hive Sink, Thrift Sink, and Avro Sink.

Flume Agent

In Flume, an agent is a daemon process that runs independently. It accepts data (events) from customers or other agents and routes it to the appropriate destination (sink or agent). Flume may contain many agents.

Flume Event

Event is the most straightforward unit of data transferred in Flume. It has a byte array payload that must be delivered from the source to the destination and optional headers.

In its most simple form, a Flume agent is a Java process comprising Source – Channel – Sink. Data is collected from the data generator in the form of Events and then delivered to the Channel. As needed, a Source can supply several Channels. The technique of a single source writing to several channels so that they can provide multiple sinks is known as fan-out.

In Flume, an Event is a unit of data transmission. Channel buffers the data until the Sink absorbs it. Sink collects data from Channel and sends it to a centralised data storage system such as HDFS, or Sink might transfer events to another Flume agent depending on the situation.

Data Flow

Flume is a platform for transferring log data into HDFS. Usually, the log server creates events and log data, and these servers have Flume agents running on them. The data generators provide the data to these agents.

The collector, an intermediary node, will gather the data in these agents. In Flume, there may be several collectors, just like there can be multiple agents. Finally, all of the data from these collectors will be combined and delivered to a centralised storage system like HBase or HDFS.

Three Types of Data Flows in Apache Flume 1) Multi-hop flow

There can be several agents in a Flume Flow. Before arriving at its final destination, the event (data) may pass through several agents. This is a multi-hop flow.


Fan-in flow

Flume allows data from various sources to be exchanged over a single channel. Fan-in flow is the term for this type of flow.

3)  Fan-out flow

Data moves from a single source to several channels in a fan-out flow. Fan-out flow may be done in two ways. They’re multiplexing and reproducing.

Relevance of Apache Flume

Flume is very useful, and there are many reasons to use Flume. Let us check some of them:

Apache Flume is the best option for transporting large amounts of streaming data from sources such as JMS or Spooling folders.

The events are staged in a channel on each agent, and the possibilities are subsequently transmitted to the following agent in the flow or a final data storage(such as HDFS). When the events have been saved in the next agent’s channel or the data storage server, they are deleted from the channel. This diagram shows how Flume’s single-hop message delivery semantics ensure the flow’s end-to-end dependability.

Flume employs a method based on transactions to ensure that events are correctly delivered. The sources and sinks encapsulate the storing and retrieval of events stored in or given by a transaction provided by the channel in marketing. This makes sure that the sequence of events is appropriately transmitted from a specific point in the flow to the next. In the event of a multi-hop flow, the earlier hop’s sink and the next hop’s source both have transactions going to verify that the data is safely placed in the following hop’s channel.

The events are staged in the channel, which controls failure recovery. Flume includes a durable file channel supported by the local file system. A memory channel saves events in an in-memory queue, which is faster, but the events that stay in the memory channel after an agent dies cannot be retrieved.

Conclusion on Apache Flume

We had a brief overview of the features of Apache Flume. To sum up:

Flume is flexible and works with various sources and sinks, including Kafka, Avro, spooling directories, Thrift, and others.

In Flume, a single source may transmit data to several channels, subsequently sending the data to many sinks, allowing a single reference to send and transfer data to multiple sinks. Fan-out is the name of this process, and Flume allows the fan-out of data.

Flume maintains a consistent flow of data transmission, which means that the data writing speed will also increase with the data reading speed.

Although Flume usually publishes data to centralised storage like HDFS or Hbase, we may configure Flume to post data to another agent if necessary. This highlights the flexible nature of Apache Flume.

Flume is a free and open-source project.

Flume is a highly adaptive, reliable, and scalable platform for sending data to a centralised storage system like HDFS. Its ability to interact with various applications like Kafka, Hdfs, and Thrift makes it a viable solution for data intake.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.


How To Track Sleep With Nest Hub And Google Fit

One of the most unique features of Google’s 2nd gen Nest Hub is Sleep Sensing. Once configured, it can deliver valuable insights into your daily sleep habits, although it’s up to you to fix them. Here’s how to track sleep with a Nest Hub and Google Fit.

See also: The best smart displays


Install a 2nd gen Nest Hub by your bedside, angled toward where your chest lays when you sleep.

In the Google Home app, go to your Hub’s settings and select Sleep Sensing. Follow instructions to set up the feature, including calibrating your sleep spot.

You can view sleep data on your Hub, or at any time on your phone using the Google Fit app.


What is Sleep Sensing for the Nest Hub?

How to setup Sleep Sensing for Nest Hub

Understanding your Sleep Summary

What is Sleep Sensing for the Nest Hub?

Jimmy Westenberg / Android Authority

Sleep Sensing uses a combination of radar, light, sound, and temperature data to track the length and quality of your sleep, along with factors that might be impacting it. It can tell if a room is unusually bright, for example, or if you (or someone else) has been snoring or coughing. Note that it’s only designed to track one person, so if you’re sleeping with a partner, they’ll need their own Nest Hub (and a separate Google account) to follow suit.

The Hub’s radar monitors movement and breathing, using that data to calculate sub-totals for Light, Deep, and REM sleep, as well as times when you’re fully awake (including getting out of bed for a few minutes). It even attempts to gauge respiratory rates.

Around your designated wake-up time, your Nest Hub delivers a Sleep Summary showing how long and well you slept. The front page of this summary is kept relatively simple, for instance rating your sleep as “restful” if it was calm or “restless” if you were constantly tossing and turning. With a few swipes, though, you can delve into more granular data, for instance highlighting that 30-minute snoring stretch at 3 AM.

All of this info syncs with the Google Fit app for Android and iOS. That allows you to catch up on the previous night if you can’t spend time in front of your Hub, as well as view long-term trends combined with data from smartwatches and fitness trackers.

Sleep Sensing is currently free for Nest Hub owners, but starting sometime in 2024, Google is planning to paywall the feature behind Fitbit Premium subscriptions. Already the feature isn’t available in all countries, or even in all languages.

How to setup Sleep Sensing for Nest Hub

In the Google Home app, tap and hold on your Nest Hub in the device list.

Tap Settings (the gear icon).

Select Sleep Sensing, then Set up Sleep Sensing.

Follow instructions to enable the feature. You’ll be offered options to track sound events and get personalized sleep suggestions from Google Fit — feel free to skip these, but you’ll be sacrificing key features.

When prompted, set a bedtime schedule. This is absolutely essential, because this tells your Nest Hub when it should run sensing, and whether or not you’re following your intended habits. Be realistic unless you’re okay with your Hub constantly rating you too late or too early.

Equally as important is calibrating your sleep spot. You should be prompted automatically, but if not, go to your Nest Hub, swipe up from the bottom edge of the screen, then tap the gear icon. Make sure Motion Sense is toggled on. Tap Sleep Sensing (accompanied by a bed icon), then Calibrate device. Position your Nest Hub so that its screen (and hence its radar) is aimed at your chest, and follow onscreen commands.

Once you’ve got your Hub calibrated, it’s best to avoid moving it unless you’re prepared to recalibrate. This can throw off results, especially if it accidentally picks up a partner, pet, or child.

Understanding your Sleep Summary

Read more: The best sleep trackers you can buy

Sonos Beam Gen 2: A Soundbar Packing A Serious Punch

The Sonos Beam is by no means cheap, but is well worth the investment for a fantastic soundbar.

The Sonos Beam is an unassuming soundbar. Small, with just a few buttons, it looks like it would be easily drowned out by a party, and struggle to keep up with a blockbuster movie or a legendary game soundtrack… but that’s just not the case.


Staying true to the Sonos brand, the Beam pumps out a loud, rich sound no matter the task you set it. Bass-heavy songs, Hans Zimmer soundtracks and lively action scenes are easily taken on, but unsurprisingly, that kind of performance doesn’t come cheap.

Setting the speaker up

As part of the packaging experience, Sonos has improved its use of eco-friendly materials. Inside the box, you won’t find any of the usual Styrofoam packaging that’s usually holding a speaker in place. Instead, Sonos uses a combination of cardboard and a nice black fabric wrapped around the Beam.

Getting the Beam set up isn’t as simple as just plugging it in. You’ll need to download the Sonos app and go through a set-up process. This includes configuring your TV, changing a few settings and connecting the speaker to the Wi-Fi.

This wasn’t the quickest process for me but after some searching on forums, it turns out that this came down purely to the very specific TV I had. Once set-up, I was able to connect my Android phone, iPad and TV, as well as activating it via Google Voice assistant (Siri is also available).

There are only a couple of ports on the speaker. Along with the charging cable input, there’s an HDMI ARC and an ethernet port.

If your TV has a HDMI ARC port, you will be able to plug the speaker straight in. If you don’t have this option (more likely on an older TV), Sonos includes an optical adaptor which allows you to connect to a different port in your TV.

If you connect via HDMI ARC, you can then control your soundbar with the volume button on your TV remote. Otherwise, the Sonos app allows you to change the volume on any connected device (like an iPhone, tablet or Android device). It is worth noting that you do need to sort the app to use this soundbar, even if you’ve plugged it into the TV.

If you buy the Beam and then decide down the line you want more of a surround sound feel, you can invest in other Sonos speakers and pair them with the Beam. A Sonos Sub and a couple of Sonos One speakers can all be paired easily to spread the sound across a room. 

The key features

One of the biggest factors that makes the Sonos Beam stand out is its versatility. Because it isn’t just a soundbar for your TV, you can connect your different devices and use the Beam to play music through your favourite streaming service.

While a lot of the best soundbars can only be used on their own, the Sonos Beam can be connected to other Sonos speakers. This means you can continually expand your home studio, adding a Sub speaker, or a couple of Sonos Ones to play sound from other points in the room.

The Sonos Beam can be set up with Google Assistant or Siri, allowing you to activate it with your voice. This can only do relatively simple commands around music unless you pair it with a streaming stick like Amazon Fire or Google Chromecast.

With certain films, songs, games and TV shows, you’ll also be able to use the Sonos Beam’s Dolby Atmos quality audio. This is a surround-sound technology that expands the height channels of your audio, making it sound like it is surrounding you, coming from above and around.

Normally, Dolby Atmos is offered in cinemas or from a full surround-sound system, but certain soundbars can condense it down. This works differently to most Dolby Atmos speakers and uses psychoacoustics to trick the listener into hearing a more impressive soundstage.

While the Sonos Beam can’t offer the same immersive experience as bigger Dolby Atmos speakers or surround sound, it does a pretty solid job for its size and price.

Apple users get the added feature with a Sonos Beam of TruePlay. This allows you to calibrate the speaker to your room using its built-in microphones. This will improve the audio for your exact set-up. However, you do need an iOS device for this. 

As the 2nd generation of this speaker, it is noticeably more expensive than its predecessor. However, that comes with a more premium build for the front grille of the speaker, better sound and connectivity, and an overhaul of the Dolby Atmos experience.

Sound experience

The most important factor of any speaker is how it sounds, and whether you’re listening to music or watching a film, the Sonos Beam excels.

Everything from the deepest bass of a song to the screech of tyres and explosions in films were crystal clear. Despite its size, the speaker packed enough power that I could at times feel the bass vibrating across the room.

As mentioned above, the Sonos Beam makes use of Dolby Atmos. For a lot of shows and films, this didn’t make a massive difference but occasionally I found it hugely improving my viewing experience.

While watching Netflix’s Drive to Survive, the Beam did a fantastic job of translating the power of the cars, engines roaring over the tense music. With Dolby Atmos, it really felt like you were there as a car crashes into a barrier at 200mph, the noise bouncing around you.

The same goes for the final fight in Avengers: Endgame. Lasers, bullets and rubble sound like they are flying past you as the ultimate battle unfolds. While a full surround system or larger speaker with Dolby Atmos will work much better, for its size the Beam utilises Dolby Atmos surprisingly well.

There doesn’t have to be lots of action for the Beam to perform. While watching the Oscar-awarded climbing documentary Free Solo, heavy breathing, rocks falling, and a tense soundtrack flew across the room in crystal clear quality.

I had a similar experience while gaming. During my tense and challenging playthrough of From Software’s Elden Ring, the soundbar captured every terrifying screech of a dragon, sword swing and the sound of my out-of-breath character running for his life – all while amplifying the beautiful soundtrack behind.

While this is designed first and foremost as a soundbar for your TV, you can play music through the Sonos Beam. Of course, this is by no means going to beat out similarly priced speakers designed purely for music, but the Beam is still an excellent performer for songs.

Unless you’re very much an audiophile trying to squeeze every drop out of your speaker, the Beam will double up as a more-than-capable speaker for music, especially when it comes to bass- and drum- heavy songs.

With Muse’s Knights of Cydonia, the horse-trot inspired drum pattern and sci-fi guitar sounded fantastic through the speaker, even when the sound was smacked up to deafening levels.

The aggressive bassline of Morning by Beck felt powerful without being overpowering, and the same goes for Thundercat’s Heartbreaks + Setbacks. Try out the pristine and well-recorded Get Lucky by Daft Punk or Radiohead’s Weird Fishes and you’ll get a blemish-free experience, enjoying the music as it was meant to be heard. 

Fitting the speaker in

Most soundbars tend to look good, especially at this price point. Sleek designs can be found across the majority of brands, but the Sonos Beam has the benefit of not being absolutely huge.

If you don’t have a large TV unit or are hoping to not take up too much room, the Beam will fit in your home better than the average soundbar. It spans a total width of just 65cm which is smaller than most mid-size to large TVs.

That means you can put it in front of the TV, on a shelf, TV unit or somewhere slightly more tucked away than larger soundbars.

It comes in either black or white. Both colours look sleek, but those who like a more unique or colourful design might be slightly disappointed.


There are a lot of soundbars out there, and the Sonos Beam sits firmly in the middle. It is by no means affordable, but it is also nowhere near the priciest one you can pick up, even within Sonos’ own range.

While it takes a few extra steps to get set up, it is an easy speaker to use from then on and offers some nice additional features like voice assistants, TruePlay for iPhone users and Dolby Atmos.

However, the Sonos Beam feels hard to critique. Music, films, games, TV shows – whatever task you throw at it, this speaker seems capable of it all. The fact that it performs so well despite its more compact size also makes this a pretty obvious choice for those with less space in their home.

Alternatives Sonos Arc

The bigger brother to the Sonos Beam, the Arc takes a lot of what works about the Beam and improves on it. Yes, it does cost a lot more at £799, but that price secures you a far larger speaker, a much more convincing Dolby Atmos experience, and obviously an improved audio experience.

If you don’t mind how much money you’re spending, there are few soundbars that can offer a better performance than this.

Sony HT-X8500

While the Sonos Beam is a great all-round soundbar, it is still quite expensive. Sony’s HT-X8500 speaker comes in at a lower price while offering an audio experience that isn’t that far behind the Beam.

It has Dolby Atmos, is easy to setup, offers a fantastic audio experience, and like the Sonos Beam, is relatively compact compared to a lot of soundbars these days.

JBL Bar 2.1

The JBL Bar 2.1 soundbar crams a lot of value into its more affordable price tag. Along with the Dolby Atmos soundbar, you also get a subwoofer to go with it. Getting this kind of combination would normally require a much larger investment.

Read more reviews:


10 Best Security Cameras That Work With Alexa And Google Assistant

When it comes to home security nowadays, the smart money is on smart cameras. Integrating your security systems with virtual assistants like Alexa or Google Assistant provides unmatched access to your home, families, and businesses while you’re away or checking up on suspicious activity in your vicinity.

But not all security cameras are created equal, and not everyone is compatible with Alexa and Google Assistant. To help you make the smart choice and find the right camera for you, we’ve rounded up some of the best security cameras with just such compatibility.

Related: 10 Best Home Security Cameras Without WiFi

Best Security Cameras that Work with Alexa/Google Assistant

Brand: Blink

Price: $34.99

Camera Type: Indoor Camera

Compatibility: Alexa

The Blink Mini is an incredibly affordable, dependable indoor camera and the most popular one within Blink’s lineup of cameras for good reason. An Amazon company, all of the Blink products automatically come with seamless Alexa integration right out of the box and a set up as easy as plugging it in, connecting it to Wi-Fi and adding it to your Blink app.

The Mini itself records 1080p video with motion detection, night vision, and 2-way audio for checking in while away. Indeed, for Alexa users looking for a perfect coupling of simplicity and affordability, the Blink Mini is certainly worth considering. 

Buy: Blink Mini

Brand: Ring

Price: $148.99

Camera Type: Outdoor Camera

Compatibility: Alexa

A paramount element in any security system is, of course, dependability. Being solar powered and IP 66 weatherproof, Ring’s new Stick Up Cam can leave you with peace of mind that, come hell or high water, your camera system will remain vigilant. The highly-rated Stick Up Cam records 1080p HD video and comes with motion detection, night vision, 2-way audio and all without the need to charge it — ever.

Another unique feature of the Ring Stick Up Cam is its customizable privacy zones for sensitive areas you don’t want to be recorded. All in all, the Ring Stick Up Cam is a solid option for those looking for a reliable, premium option from one of the biggest names in home surveillance.

Buy: Ring Stick Up Cam

Brand: Ring

Price: $199.99

Camera Type: Outdoor

Compatibility: Alexa

For those who take home defense a little more seriously, the ring floodlight camera offers extra functionality over most of its competitors. While recording 1080 P HD video with live stream capabilities, the Ring Floodlight camera eradicates blind spots in your vicinity and comes with a remote-activated siren to ward off intruders in case of emergency.

While a little more expensive, with integrated Alexa compatibility and A ring protect subscription plan, the ring floodlight camera offers possibly the most stringent home defense capabilities money can buy in a single camera. 

Buy: Ring Floodlight Cam

Brand: Google Nest

Price: $127

Camera Type: Indoor

Compatibility: Alexa, Google Assistant, Google Home

While it’s definitely a little more expensive than many other plug-in wireless cameras, the Google Nest indoor camera — unsurprisingly — offers seamless integration with both Google Assistant, Google Home, and Alexa. Specs-wise, it’s not too shabby either.

With 1080p video, motion/sound detection and night vision and a 130° wide-angle FOV, the Nest Indoor Camera delivers the essentials alongside complete reliability and excellent customer service.

Buy: Google Nest Indoor Cam

Brand: Google Nest

Price: $199.00

Camera Type: Outdoor

Compatibility: Alexa, Google Assistant, Google Home

On the ornery hand, we have the outdoor counterpart to the Google Nest Indoor Cam. In terms of features, the outdoor variant is comparable to the indoor cam, with 1080p video, night vision and sound/motion detection with a 130° FOV. What separates the two is really the weatherproofing. Hardwired to power and with exceptional build quality, Users can count on the Google Nest Outdoor Cam to stay reliable all year round.

Buy: Google Nest Outdoor Cam

Brand: Wyze

Price: $25.98

Camera Type: Indoor

Compatibility: Alexa, Google Assistant 

If the Google Nest series of cameras are a little out of your price range, the Wyze V2 offers similar functionality at a much more affordable price point. Compatible with both Alexa and Google Assistant, the Wyze Cam records at 1080p with a 110° FOV and supports 24/7 stream, motion /sound detection with 12 seconds automatic clip recording when triggered as well as 30′ night vision and 2-way audio.

So it’s feature-rich and for a price that’s hard to beat — and backed by a large and enthusiastic customer base.

Buy: Wyze Cam v2

Brand: Dragon Touch

Price: $49.99

Camera Type: Outdoor

Compatibility: Alexa, Google Assistant

Almost as badass as it sounds, the Dragon Touch OD10 comes with a potent assortment of home defense features that provide tremendous value for the price point. The areas in which the Dragon Touch notably outperforms many competitors is its 360° coverage, with 355° pan, 90° tilt, and 4x zoom, and the 66′ night vision radius — over double the standard range.

These two features, when combined with the floodlight, eradicate almost all blindspots from its FOV. On top of all that, the Dragon Touch cam offers the usual bevy of features: 1080p, motion detection, 2-way audio and IP65 weatherproofing.

Buy: Dragon Touch OD10

Brand: Teckin

Price: $39.99

Camera Type: Indoor

Compatibility: Alexa, Google Home

The best thing about the Teckin Cam 2-pack is the value for money it provides. The indoor, 1080p cam offers the now-predictable lineup of industry standards features — 32′ night vision, sound/motion detection with automatic recording, and 2-way audio — but at just about $20 per cam.

Each cam also supports local storage on a 128GB local MicroSD in case you don’t want to opt for a cloud subscription. A solid option for those looking for easy-to-set-up, dirty cheap nanny cams, pet/baby monitors, etc.

Buy: Teckin Cam

Brand: Zmodo

Price: $59.39 (about $30 per camera)

Camera Type: Outdoor

Compatibility: Alexa

The Zmodo wireless security pack offers customers a powerful security package at a more accessible price point than is usual for its features.

The standout features of the IP66 weatherproof camera are its AI-powered object recognition that differentiates between humans, pets, faces and vehicles (though this feature requires a Zmodo Cloud subscription) and the extended, 65′ night vision with smart IR-cut to drastically enhance low-light visual quality. Overall, you really can’t go wrong with the Zmodo wireless pack.

Buy: Zmodo Outdoor Security Camera 

Brand: Eufy 

Price: $149.99

Camera Type: Outdoor

Compatibility: Alexa, Google Assistant, Apple Homekit

While expensive up-front, the EufyCam 2 Pro comes with absolutely zero monthly fees and is a heck of a feature-rich camera to boot. It comes with 2K resolution, AI-powered object recognition and image enhancement, programmable activity zones for more efficient storage, and complete compatibility with Alexa, Google Assistant and Apple Homekit.

It also has the added distinctions of being 100% wireless with 365-day battery life and next-gen night vision with enhanced resolution and IR-cut. But aside from its great features, what makes the EufyCam 2 Pro great is its commitment to verifiable privacy; all data is stored and processed locally but remains accessible to you through a 256-bit encrypted transmission. No extra fees, no headaches, a whole lot of power — there’s a reason the EufyCam 2 is as popular and well-liked as it is.

Buy: EufyCam 2 Pro

Still haven’t found the security camera you need? Let us know the kind of specs/features you’re looking for or the unique security needs you’re trying to meet — we’d love to help!

Update the detailed information about Etl Pipeline With Google Dataflow And Apache Beam on the website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!