Usage of Redis

I must have had my head in the sand for too long, and until this thread had not heard of Redis (aka Remote Dictionary Server). I didn’t want to hijack that thread so am starting a new one. I read over some Redis documentation, and it strikes me as being potentially very useful where I was otherwise keeping some dictionaries or JSONItems stored in string format for persistence. And especially where I was trying to make those dictionaries accessible to another process on the same machine (via sockets), and to a lesser extent another thread within the same app (via scoped dictionaries or classes which use a dictionary/JSONItem at heart).

I also downloaded a copy of Kem’s library for connecting to a Redis server. (Thanks!)

A few questions about Redis for those who have gone down this path before:

  • Are there any known limitations to using macOS as the Redis server? The website seems to prefer Linux but when I did a local install on my MBP it appears to run fine using tutorial sample commands and Kem’s unit tests.
  • Is there a stable version of Redis 4.x server for Windows? For my current projects where I could see considering Redis, I would not need Windows as a server but I’m just trying to understand future limitations if I start considering this technique beyond some immediate projects I have in mind.
  • For macOS, is it reasonable to bundle a redis-server executable and redis.conf file inside your own app bundle then use a custom port to keep from conflicting with another potential redis server on the same network? For example, for usage as mentioned in this thread for the app’s own private use. Or in a desktop app which uses console helper apps in lieu of internal threads. Even if this works for third-party installations, would it run afoul of allowing an app installer to be signed?
  • Is Redis a useful technique for inter-process communication between apps on the same localhost? There are times I have rolled my own TCP or UDP sockets instead of using IPC, where the secondary process is mostly read-only. For example a “watchdog” process that silently monitors a critical process to help ensure high availability. Or a GUI monitor that displays consolidated status/progress information from multiple console processes.
  • Is Redis Publish/Subscribe messaging paradigm helpful here for real-time notifications and events instead of polling for data or using BLPOP to block and wait for a list entry?
  • With proper port forwarding and authentication, is it feasible for the Redis server to really be remote, as in not in the same LAN? Or is it better to use something like Aloe Express to expose an API for requesting data even if that data ends up being equivalent to a Redis hash entry?

Redis just seems to have quite a bit of very useful non-relational capabilities already developed, optimized, and tested. Trying to decide how much time to put in sample projects to learn if that is worth exploiting.

I’ll try to answer what I can. Some of this is anecdotal.

  • No limitations on MacOS that I can find. The Mac version is being kept fairly up to date, at least through MacPorts.
  • AFAIK, the Windows version is stuck at an older version for now, but it really shouldn’t matter for simple use.
  • I haven’t tried bundling, but that seems quite reasonable, and you should be able to run that through a shell if nothing else. That way, if your app dies, your custom Redis will die with it.
  • We are turning to Redis for IPC ourselves, and here’s why: we run several instances of our middleware and they all need common information. The internal IPC implementation is insufficient because it’s limited to one-to-one communication. Instead, the utility app updates a known key with its status in JSON form and the other apps read that status as needed. We also set that status to expire after x minutes so, if the utility app dies unexpectedly, the useless status doesn’t stick around for too long. Finally, we are storing the utility app’s pid in a separate key so we know which instance to kill if needed. This is all new so it hasn’t gone into production yet, but I’m not anticipating any issues.
  • I have not fully explored publish/subscribe.
  • Redis is designed for remote as well as local access, but take security issues into mind. You can set up a password for access in the Redis config, or on the fly, and limit access to that port from known machines.

HTH.

Kem,

Thanks for the reply.

Regarding IPC, I think we are in the same boat. But the idea of setting an expiration is interesting. I had thought I’d just include a timestamp property and test that it was not too old. But self-expiring has its own virtues too.

It appears that your M_Redis module has no external dependencies on a Redis client, right? So if the an app only needs a client connection it works with no external dependencies? And that suggests it is cross-platform if you needed a Windows machine as a client?

Doug

Yes. In fact, I built in some code to handle the older version intelligently. For example, if you call Delete on my class, it calls UNLINK if the server supports it, or DEL if not.

@Douglas Handy , I wrote a proof-of-concept for bundling Redis server with an app, and it works great, at least on the Mac. I’d expect it to work fine with other platforms too.

In short, my app starts Redis server in an asynchronous shell, then uses my class to SET and GET a value, then shut the thing down. It’s not pretty code, but it works. I’d be happy to share it if you’re interested and might include it as part of the repo.

In fact, maybe I should include a RedisServer class that takes a path to the server, the config file and any parameters to start a local server. Hmmm…

@Kem Tekinay, I decided to run the unit tests against a Redis server across the net. I setup the server on a macOS install, setup the port forwarding and password, etc. to see what would happen. Speeds to each respective ISP were:

  • Server: 6ms ping, 89Mbps down, 93Mbps up
  • Client: 11ms ping, 125Mbps down, 24Mbps up
  • Client to server route: 11 hops
  • Client to server ping 10 times: round-trip min/avg/max/stddev = 23.060/31.649/48.721/8.431 ms

So averaged about 32ms ping time, which obviously can impact the unit tests. The tests which always failed for me were the Decrement and Increment variants. For example:

  • Decrement : Expected [3] but was [-1].
  • DecrementBy : Expected [1] but was [-3].
  • Increment : Expected [5] but was [1].
  • IncrementBy : Expected [10] but was [6].

The GetLatencyReport passed with
Samples: 41
Total: 1.002 s
Min: 23.286 ms
Max: 29.038 ms
Avg: 24.441 ms

Some others failed sometimes and not other times. So I think some unit tests are sensitive to timing. The actual benchmark tests locked up on me and would not finish. I have not researched why yet.

I like the idea of a RedisServer class…

Is this a server I can access too?

Yes, I will send you a PM with credentials.

Cool, thanks. I’ll check it out here.