CoPrA – An Asyncronous Python Websocket Client for Coinbase Pro

I know it’s too much too hope that I have any regular readers, but if you  come here even infrequently you may have noticed it’s been seven months since I last posted. You may even have wondered what I have been up to.  What could’ve possible kept Tony so busy that he would totally ignore his blog for more than half a year? Well, I’ll tell you. Fortnite. I have been busy getting my ass kicked 24-7 by eleven year-olds in Fortnite. It is embarrassing quite honestly.

In between matches, however, I actually have been productive working on intelligent agents for trading cryptocurrency on the Coinbase Pro (formerly GDAX) exchange. While most of the high level code isn’t ready for prime time yet, a good portion of the low level, workhorse code has been road tested and is relatively stable.  Since other developers might find it useful, I’ve decided to begin releasing it into the wild.

This week I released the first of my code: CoPrA, an asyncronous Python WebSocket client.  CoPrA is built on top of the phenomenal Autobahn|Python WebSocket framework.  While it is certainly possible to use Autobahn itself for a Coinbase client, its options are multitudinous and generally more complicated than necessary for most applications.  My goal with CoPrA, therefore, was to simplify the WebSocket interface without sacrificing any of the functionality I need.

I will be following up shortly with a more detailed tutorial for using CoPrA, but if you’d like to take it for a test drive now, you can install it using pip:

$ pip3 install copra

CoPrA can also be installed from source. Either clone the repository from github:

$ git clone https://github.com/tpodlaski/copra

or download the tarball:

$ curl -OL https://github.com/tpodlaski/copra/tarball/master'

Once you’ve downloaded a copy of the source files, you can install CoPrA with:

$ python3 setup.py install

I’m looking for feedback, suggestions, bugs, or any other comments, so please feel free to leave them in the comments below.  Look for another post on CoPrA soon.

Links:

27 comments

Skip to comment form

  1. Awesome… will try this weekend! Played with another library but it gets clogged up calling blocking functions and I’d like to avoid threads if possible.

      • Tony on September 20, 2018 at 1:30 pm
        Author
      • Reply

      I’d love to hear out it works out for you. Good luck!

  2. Don’t want to mess up your github with dumb questions… in the on_message callback, I occasionally need to call a long-running function (I’m placing a buy/sell limit order at best ask/bid and canceling/replacing it if price moves – to avoid paying maker fees and this can take a few seconds or even several minutes) and this stops me from getting ws messages until the function returns.

    Is there a recommended way to asynchronously call my long-running function so I keep getting on_message callbacks?

    Thanks.

      • Tony on September 21, 2018 at 5:39 pm
        Author
      • Reply

      I would suggest making your long running function asynchronous and then use loop.create_task to add it to the loop where you need to in on_message. This will run your function in “parallel” with the rest of the websocket code.

      1. Great, I was just starting to try that!!

      2. I know you didn’t sign up to debug my code, but…

        I’ve tried:
        loop = asyncio.get_event_loop()
        task = loop.create_task(update_candles(parsed_time, num_candles=1, quote_curr=’ETH’, base_curr=’USD’))

        also tried:
        asyncio.ensure_future(update_candles(parsed_time, num_candles=1, quote_curr=’ETH’, base_curr=’USD’))

        def async update_candles()

        In both cases, it calls the update_candles() function but blocks

          • Tony on September 21, 2018 at 6:42 pm
            Author
          • Reply

          update_candles will block once the loop calls it unless you’re able to await whatever is holding you up. Are you waiting for a state change? There are a few (more elegant) ways to do it, but I usually just set up a while loop that checks the condition and exits when the condition is met or otherwise calls await asyncio.sleep(0) which releases control back to the event loop for a cycle before checking the condition again.

          1. update_candles() does this:
            * makes buy/sell decision
            * gets latest bid/ask (REST, but want to use ws ticker)
            * places buy or sell at best bid or ask (REST)
            * loops and gets order status
            – If filled, then return
            – if bid/ask changes then cancel existing order and replace at latest bid or ask
            – else continue

            There is no async code within update_candles.

            • Tony on September 21, 2018 at 7:12 pm
              Author

            I’ve written similar code although I am using an asynchronous FIX client for placing/cancelling orders. I’m pulling the asks/bids and order statuses from the websocket feed. The bottleneck for me was watching for the bid/ask changes. That was the loop I put await.asyncio.sleep(0) in.

          2. Oh, I see… ok, let me try to insert some await asyncio.sleep(0) in the loop

          3. Awesome… adding a bunch of await asyncio.sleep(0) seems to do the trick and the main blocker now is just a synchronous REST call to get bid/ask… but I can rewrite that to get it from ws ticker and so things should be rocking now. Thanks so much. When I get this working, there will be invite to the private island it buys. 😉

      3. And the happy ending is that slippage is down by like 75% using ws instead of rest. Thanks again.

          • Tony on September 22, 2018 at 4:59 pm
            Author
          • Reply

          You are very welcome. I’m very happy it is working out for you. If you have any other questions or just want to talk shop, shoot me a message. Best of luck!

      • Tony on September 21, 2018 at 5:53 pm
        Author
      • Reply

      On a side note, I have code for an asynchronous REST client for Coinbase Pro that I am cleaning up, writing tests and documentation for, etc. that I plan to add to CoPrA. No ETA on that unfortunately.

      1. No worries… build one for Bitmex! afaik, they have no working Python library and don’t seem to care that much about.

          • Tony on December 1, 2018 at 4:49 pm
            Author
          • Reply

          As of version 1.1.0, the CoPrA package has an asynchronous REST client. It only took 2 months, but it’s fully tested, documented, and live now.

    • Charles on January 8, 2019 at 11:54 pm
    • Reply

    Thanks for a thoughtful library!

      • Tony on January 9, 2019 at 8:03 am
        Author
      • Reply

      You are welcome! If you have comments, questions, or problems please let me know.

    • Marko on January 27, 2020 at 2:34 am
    • Reply

    Awesome Library man. I’ve been playing with it here and there and I’ve noticed that sometimes Coinbase Pro will close with a code 1006 (Abnormal connection). Have you noticed this or known to occur before?

      • Tony on February 9, 2020 at 6:04 pm
        Author
      • Reply

      I actually haven’t noticed that before. I’ve had some pretty long-running Websocket connections to Coinbase and only a few disconnects. They have never been frequent enough or persistent enough to motivate me to track them down. If you figure anything out, please feel free to share.

        • Marko on February 10, 2020 at 1:25 am
        • Reply

        Yea, I’ve figured out I was not implementing on_close() properly as it has a reconnect function that fixes the 1006 disconnection issue. I don’t know why it gets disconnects or if any data is lost in between auto reconnects. I just recently realized my error so I haven’t had the time to look into it. I did realize that txid can be counted sequentially to find any holes. Anyways.. All in all pretty cool – its running long now. I will definitely track it to ground when I have time.

          • Marko on February 10, 2020 at 1:30 am
          • Reply

          Oh Tony, I recall now there is a ping interval that could cause disconnects. Does the autobahn package have any pings it does to the websocket to keep it open? Here is a SO for the websockets library: https://stackoverflow.com/questions/54101923/1006-connection-closed-abnormally-error-with-python-3-7-websockets

          See first answer.

            • Tony on February 10, 2020 at 6:24 am
              Author

            Marko, autobahn does provide for a configurable ping. I should probably document this better, but for now take a look at https://github.com/tpodlaski/copra/issues/6 and see if that helps you at all. As far as that goes, you can adjust any of the autobahn protocol options using client.setProtocolOptions. A good summary of the available options is here: https://autobahn.readthedocs.io/en/latest/websocket/programming.html#websocket-options. I hope this helps.

  3. Hi, I’m passing an array called quotepair on to the client.
    quotepair=[‘BTC-USDC’,’ETH-BTC’,’DAI-USDC’,’XLM-USD’,……………..
    ws = Client(loop, Channel(‘ticker’, quotepair))
    The plan is to get a stream of prices for each pair.
    I am getting this stream.
    {‘type’: ‘ticker’, ‘sequence’: 113942752, ‘product_id’: ‘XLM-BTC’, ‘price’: ‘0.00000719’, ‘open_24h’: ‘0.00000715’, ‘volume_24h’: ‘5148029.00000000’, ‘low_24h’: ‘0.00000700’, ‘high_24h’: ‘0.00000735’, ‘volume_30d’: ‘155312439.00000000’, ‘best_bid’: ‘0.00000719’, ‘best_ask’: ‘0.00000721’, ‘side’: ‘sell’, ‘time’: ‘2020-02-09T02:28:37.816461Z’, ‘trade_id’: 821112, ‘last_size’: ‘280’}
    etc, etc, etc.
    Which is good, but it streams onto the console. I’m trying to populate an array of [pair,prices]
    How do I tame this stream?
    I don’t want it streaming on to the console and I want to populate an array.
    Thanks

      • Tony on February 8, 2020 at 10:27 pm
        Author
      • Reply

      Hi Bob,

      You need to subclass copra.rest.Client.

      The Client class has 4 callback methods that are called automatically during the client’s life cycle. The method on_message is called every time a new message is received. The default implementation simply prints the message to stdout.

      If you want to populate an array, subclass copra.rest.Client and override on_message. The ticker messages are the Python dictionaries you are seeing in the console, so in on_message you can inspect the message dictionary and do whatever you want with the data.

      You can read about the Client callback methods here: https://copra.readthedocs.io/en/latest/websocket/usage.html#callback-methods.

      Hope this helps. Let me know if you have any more questions.

  4. Yes, it helps immensely, was able to improve my polling time for all my pairs from 7.8 seconds to 0.001, that’s a 7800x improvement. Now I just have to figure out how to get the loaded price array into my algo.

      • Tony on February 9, 2020 at 4:48 pm
        Author
      • Reply

      I am glad to hear you got it working. I found the best way to move data around ayncronously is with Asyncio Queues. Have the client’s on_message method put the data in the queue and build an ansync method connected to your algorithm that checks the queue for new data on each pass through the event loop .

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.