Everand Logo

Welcome to Everand!

  • Discover millions of ebooks, audiobooks, and so much more with a free trial

    Only $11.99/month after trial. Cancel anytime.

    Instant Parallel Processing with Gearman
    Instant Parallel Processing with Gearman
    Instant Parallel Processing with Gearman
    Ebook108 pages44 minutes

    Instant Parallel Processing with Gearman

    By John Ewart

    ()

    Read preview

    About this ebook

    In Detail

    Many of today's applications need to be able to process large volumes of data, and vertical scaling has its limits both in terms of prohibitive cost and hardware limits. Gearman is an open source job manager that is well-suited to building horizontally scalable systems, from map-reduce algorithms to simple data processors capable of handling massive amounts of information.

    Instant Gearman is a practical, hands-on guide to getting started with building an open source job management server system that is built to grow. Learn the basics of building a distributed application that spans multiple components and learn how Gearman fits into building an application that scales from one to hundreds of components that interact to process data. With Gearman, you can build software that scales horizontally as your need for computation increases.

    Instant Gearman has in-depth examples and a step-by-step approach to building distributed systems, helping you to build systems that are scalable and modular in their approach to processing data.

    Once you are comfortable with building simple workers and clients, learn how to build a cluster of managers and see how to reduce single-point-of-failure in your architecture. Next, build a simple map-reduce application using Gearman and scale it up from a single instance to multiple parallel processing components.

    Approach

    Get to grips with a new technology, understand what it is and what it can do for you, and then get to work with the most important features and tasks. This is a quick tutorial allowing you to quickly increase the efficiency of your system by intelligently reducing its workload where possible and learn techniques for designing your solutions to take advantage of Gearman's architecture.

    The technology covered in this book is usable by almost any modern programming language and any type of application, be it web, desktop, or mobile applications.

    Who this book is for

    This book targets developers who are interested in adding distributed data processing to their applications. The examples in this book are written using Ruby, but are documented thoroughly to allow developers comfortable with other languages to understand how to use them in a language of their choice. Intermediate programmers with no previous experience in distributed computing will also be able to easily follow the content.

    LanguageEnglish
    PublisherPackt Publishing
    Release dateJul 26, 2013
    ISBN9781783284085
    Instant Parallel Processing with Gearman
    Author

    John Ewart

    John Ewart is a systems architect, software developer, and lecturer. He has taught courses at a variety of institutions including the University of California, California State University, and local community colleges, covering a wide range of computer science topics including Java, data structures and algorithms, operating systems fundamentals, UNIX and Linux systems administration, and web application development. In addition to working and teaching, he maintains and contributes to a number of open source projects. John currently resides in Sonora, California with his wife, Mary, and their two children.

    Related to Instant Parallel Processing with Gearman

    Related ebooks

    Programming For You

    View More
    View More

    Related podcast episodes

    Related articles

    • Article

      Password Managers

      Feb 6, 2024

      Password Managers
      14 min read

    • Article

      FLASK Web Frameworks

      Jun 4, 2019

      The main focus of Python has always been to get you cracking on with your coding – the language was never made for web programming. However, this has just made it more interesting to extend the language for the web, or to create an interface to web-b

      9 min read

    • Article

      Lag Is Killing Games

      Jan 11, 2022

      Lag Is Killing Games
      8 min read

    • Article

      Usability

      Oct 19, 2021

      Usability
      3 min read

    • Article

      Web Browsers

      Jul 26, 2021

      Web Browsers
      11 min read

    • Article

      Ditch The Filing Cabinet

      Aug 10, 2023

      Ditch The Filing Cabinet
      3 min read

    • Article

      Ditch the MONTHLY PAYMENTS

      Dec 9, 2021

      Ditch the MONTHLY PAYMENTS
      7 min read

    • Article

      Smartproxy

      Jan 9, 2024

      Smartproxy
      2 min read

    • Article

      Share Photos With Clients

      Mar 21, 2023

      Share Photos With Clients
      2 min read

    • Article

      Poisoning The Well

      Jan 11, 2022

      Poisoning The Well
      4 min read

    • Article

      Hot Picks

      Mar 9, 2021

      Hot Picks
      13 min read

    • Article

      Choices, Choices

      Apr 5, 2022

      Matt Yonkovit is the head of Open Source Strategy at Percona “Many modern programs are built with dozens of different open source components, constructed like LEGO from pre-built blocks. This approach to picking and choosing the best tools and compon

      1 min read

    • Article

      Choices, Choices

      Apr 5, 2022

      Matt Yonkovit is the head of Open Source Strategy at Percona “Many modern programs are built with dozens of different open source components, constructed like LEGO from pre-built blocks. This approach to picking and choosing the best tools and compon

      1 min read

    • Article

      Mac Writing Apps

      Nov 15, 2022

      Mac Writing Apps
      5 min read

    • Article

      Craft A Perfect Personal Document Library

      Jan 26, 2023

      There are countless apps available that can be used to organise your notes and also many word processors designed to help you create smart-looking documents, but Craft aims to do both in style. The idea is to include a huge number of advanced documen

      1 min read

    • Article

      The Cost Of Creativity Crisis

      Oct 6, 2022

      With inflation at a forty-year high, we’re surrounded by rising prices. And when it comes to the business of writing, the cost of creativity is increasing, too. There’s also a growing trend in the software industry to move away from one-time purchase

      6 min read

    Related categories

    Reviews for Instant Parallel Processing with Gearman

    0 ratings

    0 ratings0 reviews

    What did you think?

    Tap to rate

    Review must be at least 10 words

      Book preview

      Instant Parallel Processing with Gearman - John Ewart

      Table of Contents

      Instant Parallel Processing with Gearman

      Credits

      About the Author

      About the Reviewer

      www.packtpub.com

      Support files, eBooks, discount offers and more

      packtlib.packtpub.com

      Why Subscribe?

      Free Access for Packt account holders

      1. Instant Parallel Processing with Gearman

      So, what is Gearman?

      Distinguishing features of Gearman

      Overview of components

      The conversation between the actors

      Usecase – image processing

      Quick Start – building your first components

      Step 1 – running a server

      Step 2 – downloading a client library

      Your first client

      Writing the client

      Running your newly created client

      Verifying that it was submitted

      Step 3 – our first worker

      Running your newly created worker

      Step 4 – varying priorities

      Step 5 – background tasks

      And that's it

      Top 5 features you need to know about

      Job handling

      Completed versus successful

      Unique identifiers and coalescing

      The client

      The worker

      Making it work

      Scaling your system

      Running multiple managers

      Writing a client that supports multiple managers

      Submitting jobs to multiple managers

      Handling a manager being terminated

      Processing jobs from multiple managers

      MapReduce

      The shared library

      A simple program

      The client

      The worker

      Parallelizing the pipeline

      Scaling this solution

      Processing a large data file

      Providing job updates

      Data updates

      The worker

      The client

      Status updates

      The worker

      The client

      Building a processing pipeline

      Using multiple languages

      Persistence engines

      Why is this important?

      Persistent versus non-persistent

      How safe is safe?

      People and places you should get to know

      Client libraries

      Community

      Instant Parallel Processing with Gearman

      Instant Parallel Processing with Gearman

      Copyright © 2013 Packt Publishing

      All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

      Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

      Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

      First published: July 2013

      Production Reference: 1230713

      Published by Packt Publishing Ltd.

      Livery Place

      35 Livery Street

      Birmingham B3 2PB, UK.

      ISBN 978-1-78328-407-8

      www.packtpub.com

      Credits

      Author

      John Ewart

      Reviewer

      Josh Black

      Acquisition Editor

      Vinay Argekar

      Commissioning Editor

      Priyanka Shah

      Technical Editor

      Mausam Kothari

      Project Coordinator

      Michelle Quadros

      Proofreader

      Bernadette Watkins

      Graphics

      Ronak Dhruv

      Production Coordinator

      Prachali Bhiwandkar

      Cover Work

      Prachali Bhiwandkar

      Cover Image

      Prachali Bhiwandkar

      About the Author

      John Ewart is a systems architect, software developer, and lecturer. He has designed and taught courses at a variety of institutions including the University of California, California State University and local community colleges covering a wide range of computer science topics including Java, data structures and algorithms, operating systems fundamentals, UNIX and Linux systems administration, and web application development. In addition to working and teaching, he maintains and contributes to a number of open source projects. He currently resides in Redmond, Washington with his wife, Mary, and their two children.

      About the Reviewer

      Josh Black has been working with computers professionally for 20 years. He has a broad range of experience and expertise including systems and network administration, mobile app development, and production web applications. Josh earned a BS in computer science with a minor in math from California State University, Chico, in 2005. He currently resides in Chico, California, with his wife Rachel, and their four children.

      www.packtpub.com

      Support files, eBooks, discount offers and more

      You might want to visit www.packtpub.com for support files and downloads

      Enjoying the preview?
      Page 1 of 1