Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request - lazy redis connection for JsonModel/HashModel #524

Open
svabra opened this issue Jun 17, 2023 · 1 comment
Open

Feature Request - lazy redis connection for JsonModel/HashModel #524

svabra opened this issue Jun 17, 2023 · 1 comment

Comments

@svabra
Copy link

svabra commented Jun 17, 2023

ARCHITECTURE ANTI-PATTERN
Declaring a datasource with the model is tightly coupling the two. Here an example of the Meta class added to a JsonModel/HashModel

REDIS_DATA_URL = os.environ.get("REDIS_OM_URL", 'redis://localhost:6380')
[..]
class Meta:
        global_key_prefix="performance-validation"
        model_key_prefix="order"        
        database = get_redis_connection(url=REDIS_DATA_URL, decode_responses=True)

ARGUMENT
There is one dominant use cases when this becomes a problem at runtime in particular. You are using one or several JsonModels/HashModels in an application, you start the application but the redis server is not yet there or not yet ready. The application:

  • startsup ...
  • the imports are loaded (also the JsonModels/HashModels one wants to use)
  • the Models will try immediately to connect to redis and fail

The redis.Redis([..]) connection attemp can be handled with a @Retry i.e. using the tenacity library. However, NOT the models. So the models will actually fail the applications. That CANNOT be a desired outcome. In distributed computing there is always the possibility that a service (like redis) is not available for whatever reason.

Furthermore, this also means, one cannot run github actions (or any other CI/CD test) unless one pulls up a redis service before testing. Yes, feasible, but not desirable.

SUGGESTION

  • invoke the connection attempt when one uses the model (e.g. before the save() method is invoked). Add a method connect
  • if there really is an advantage of a immediate connection at import-time, then we could implement eager and lazy-connection attempts.

(I do understand one can still use the Redis/StrictRedis object to "manually" add record to any of the redis instances. Anyhow, the redis-om-pyhton library is really the way to go. A very neat piece of software so far. Thanks to all who contribute to this library.)

REFERENCES
Apparently I'm not the only one disliking the tight-coupling: #519

@svabra svabra changed the title Feature Request - multi Feature Request - multi redis instances for JsonModel/HashModel Jun 17, 2023
@svabra svabra changed the title Feature Request - multi redis instances for JsonModel/HashModel Feature Request - lazy redis connection for JsonModel/HashModel Jun 24, 2023
@brettbeeson
Copy link

brettbeeson commented Jul 16, 2023

Hi!

I'd like to add a (common?) use-case for this. We use a cache as a 'nice to have' and would like it to silently failover to doing nothing. In other words, if there is a ConnectionError, just treat all .get() by raising NotFound. The application continues, just without caching. This concept is well-described here (Django context, but same idea).

I've done a proof of concept for my purposes, which might interest some.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants