Zmq in context of openstack

71 %
29 %
Information about Zmq in context of openstack

Published on March 6, 2014

Author: yatinkumbhare



ZMQ socket based networking library, which can also be molded as broker-less messaging framework. The slides talks about how zmq been woven to work with openstack services.

Openstack Messaging with ZMQ (Distributed RPC) By Yatin Kumbhare

ZMQ / 0MQ / ZeroMQ -Introduction ● Zero Broker, Zero latency, zero cost - culture of minimalism. ● Brokerless - No SPOF ● High Scalability ● Flexible: inproc, ipc, tcp ● Order of bind and connect - Doesn’t matter ● Fast and reliable

REQ-REP # Client import zmq context = zmq.Context() print("Connecting to hello world server…") socket = context.socket(zmq.REQ) socket.connect("tcp://localhost:5555") for request in range(10): print("Sending request %s …" % request) socket.send("Hello") # Get the reply. message = socket.recv() print("Received reply %s [ %s ]" % (request, message)) # Server import zmq context = zmq.Context() # Socket to talk to server print("Connecting to hello world server…") socket = context.socket(zmq.REP) socket.bind("tcp://*:5555") while True: print("waiting for client request") msg = socket.recv() print("Message received %s “ % msg) message = socket.send("World")

PUB-SUB import zmq from random import randint context = zmq.Context() socket = context.socket(zmq.PUB) socket.bind("tcp://*:5556") import zmq import sys context = zmq.Context() socket = context.socket(zmq.SUB) socket.connect("tcp://localhost:5556") while True: zipcode = randint(1, 10) temperature = randint(-20, 50) zip_filter = sys.argv[1] if len(sys.argv) > 1 else “” humidity = randint(10, 15) print “Publish data:”,(zipcode, temperature, humidity) socket.send("%d %d %d" % (zipcode, temperature, relhumidity)) print “Collectin updates from weather service %s” % zip_filter socket.setsockopt(zmq.SUBSCRIBE, zip_filter) #process 5 records for record in range(5): data = socket.recv() print data


PUSH-PULL # Ventilator # socket to send messages on sender = context.socket(zmq.PUSH) sender.bind( “tcp://*:5557” ) sink = context.socket(zmq.PUSH) sink.connect( “tcp://localhost:5558” ) sink.send( “0”) # Workers: receiver = context .socket(zmq .PULL) receiver .connect( "tcp://localhost:5557" ) # Socket to send messages to to sink sender = context .socket(zmq .PUSH) sender.connect( "tcp://localhost:5558" ) # Sink: # Socket to receive messages on receiver = context .socket(zmq .PULL) receiver .bind("tcp://*:5558" ) # Wait for start of batch s = receiver .recv()



ZMQ for Openstack ● In openstack, zeromq makes use of two protocols, TCP and IPC. ● TCP Socket, for sending message from nova services to rpc-zmq-receiver service. ● IPC Socket, for forwarding message received by rpc-zmq-receiver to the local nova services, which is listening on IPC socket. ● ZMQ uses PUB-SUB and PUSH-PULL socket types. ● PUB-SUB - to get reply, in case of, this work as direct-consumer/publisher in context of rabbitmq. ● PUSH-PULL - for sending messages to other services, for example rpc.cast.

zmq-receiver service ● rpc-zmq-receiver service with different sockets. ● Instantiate ZmqProxy class from oslo.messaging, impl_zmq. py file. ● Service creates TCP://*:9501, PULL socket, and listen for messages from other services. ● Topic: fanout~service or zmq_replies.<hostname>, zmq.PUB socket ● Topic: service.<hostname>, (conductor.node) zmq.PUSH socket is created. ● The socket protocol will be IPC. ● Here, data is received on TCP://*:9501 and forwarded to IPC socket.

Nova-* services ● Instantiate ZmqReactor from oslo.messaging, file. ● Mostly the nova service manages with zmq.PUSH, zmq.PULL and zmq.SUB socket type. ● For consuming messages nova service uses IPC sockets with zmq.PULL, and to cast a message it creates TCP, zmq.PUSH socket. ● example: when zmq-receiver, publish message on ipc: ///var/run/openstack/zmq_topic_conductor.node, with PUSH socket, which is consumed by ipc: ///var/run/openstack/zmq_topic_conductor.node with PULL socket at conductor service.

Nova-compute ● topic: fanout~compute, bind to ipc:// , zmq.SUB socket ● topic: compute.<hostname>, bind to ipc:// , zmq.PULL ● nova compute does on conductor at the start of the service. ● The hostname for conductor service is read out from MatchMakerRing file. ● compute service, creates tcp://<conductor-hostname>:9501, PUSH socket and send message, which is received by zmqreceiver, and forwarded to nova-conductor service. ● As, this is, to receive reply, nova compute create, ipc: ///var/run/openstack/zmq_topic_zmq_replies.<hostname> with SUB socket type and subscribe to msg-id (unique uuid), and await for response from conductor.

MatchMakerRing file { "conductor": [ "controller" ], "scheduler": [ "controller" ], "compute": [ "controller","computenode" ], "network": [ "controller","computenode" ], "zmq_replies": [ "controller","computenode" ], "cert": [ "controller" ], "cinderscheduler": [ "controller" ], "cinder-volume": [ "controller" ], "consoleauth": [ "controller" ] }

Message routing compute make to conductor. Sends data on tcp://gravity:9501 with PUSH, which is received by zmq-receiver service of gravity host with topic:conductor. gravity 'conductor' {'args': {'service': {u'binary': u'nova-compute', u'created_at': u'2014-0219T11:02:32.000000', u'deleted': 0, before, include following data ● reply-topic as zmq_replies.nodex u'deleted_at': None, ● create unique msg_id, u'disabled': False, ● extra key "method":"-reply" u'disabled_reason': None, u'host': u'nodex', To receive reply on topic:zmq_replies.nodex, creates socket ipc://...zmq_topic_zmq_replies.nodex with SUB and subscribe to msg_id. u'id': 2, u'report_count': 22, u'topic': u'compute', The conductor consume data and takes out reply-topic as zmq_replies.nodex Conductor sends back the response by creating tcp://nodex: 9501 with PUSH and send data. u'updated_at': u'2014-0219T11:08:10.000000'}, 'values': {'report_count': 23}}, 'method': 'service_update', 'namespace': None, 'version': '1.34'}

Message routing ● nodex zmq-receiver service, receiver data and check for topic, based on topic which is u'zmq_replies.nodex' {'args': {' msg_id': u'103b95cc64ab4e39924b6240a8dbaac8', 'response': [[{'binary': u'nova-compute', zmq_replies.nodex, create ipc://.... /zmq_topic_zmq_replies.nodex with PUB and 'created_at': '2014-0219T11:02:32.000000', send data. ● 'deleted': 0L, Now, the compute already has reply socket ipc:// 'deleted_at': None, with SUB, and is already subscribe to msg_id, 'disabled': False, which will get back the response of 'disabled_reason': None, 'host': u'nodex', 'id': 2L, 'report_count': 23, 'topic': u'compute', 'updated_at': '2014-0219T11:08:33.572453'}]]}, 'method': '-process_reply'}

Devstack with zmq #localrc ● ENABLED_SERVICES+=,-rabbit,-qpid,zeromq ● #controller ENABLED_SERVICES=n-cpu,n-net,zeromq #compute node Apply Patch: Due to an issue with zeromq support for notifications in Glance, glance-api fails to start when configured to use ZeroMQ. Commnet line in lib/glance in devstack repository #iniset_rpc_backend glance $GLANCE_API_CONF DEFAULT

#process presentations

Add a comment

Related presentations

Related pages

ZeroMQ - OpenStack

... each host (running OpenStack services) is both ZeroMQ client ... // In Progress Create zmq context once ...
Read more

ZeroMQ configuration assistance [closed] - Ask OpenStack ...

... matchmaker_ring.MatchMakerRing rpc_zmq_port=9501 rpc_zmq_contexts=1 rpc_zmq_ipc_dir=/var/run/openstack rpc_zmq_host=openstack5 rpc_backend=zmq The ...
Read more

ZeroMQ Context should be created once per driver instance ...

ZeroMQ Context should be created once ... default_exchange=None, allowed_remote_exmods=None): self.ctxt = zmq.Context (CONF.rpc_zmq ... OpenStack ...
Read more

openstack/oslo.messaging - OpenStack library for messaging

... ... diff --git a/oslo_messaging/_drivers/zmq_driver/client/ b ... def __init__(self, context ...
Read more

Configure the Oslo RPC messaging system - OpenStack ...

Configure the Oslo RPC messaging system. ... rpc_zmq_contexts = 1 (IntOpt) ... rpc_zmq_ipc_dir = /var/run/openstack
Read more

Oslo/GraduationStatus - OpenStack

Oslo/GraduationStatus < Oslo. ... openstack/cliff repository Depends on: ... zmq M: Mike Wilson M: ...
Read more

Change ZeroMQ driver underlying pattern to REQ/REP ...

Change ZeroMQ driver underlying pattern to REQ/REP ... // ... OpenStack Foundation.
Read more

Zmq | LinkedIn

View 75 Zmq posts, presentations, experts, and more. Get the professional knowledge you need on LinkedIn. LinkedIn Home What is LinkedIn? Join Today
Read more