Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shopify.Pagination module #92

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

balexand
Copy link
Contributor

Fixes #83. See the doc comments in the Shopify.Pagination module for details and examples.

@balexand
Copy link
Contributor Author

balexand commented Jan 1, 2020

It's possible that the :middleware option can be removed in order to simplify things. A custom adapter could be used to throttle requests instead. Please let me know if you have any preferences. Happy new year!

@larskluge
Copy link
Contributor

Thanks @balexand for your contribution. @nsweeting any chance you have some feedback on this for us all? Shopify will remove for page based navigation in less than a month now: April 1st. Thank you both!

@balexand
Copy link
Contributor Author

balexand commented Mar 6, 2020

Let me know if there's anything that I can do to help. I'm currently using the code from this PR in production and it works great.

@ProtoJazz
Copy link

@balexand since this doesn't seem to be getting merged in I've been using your fork. I wonder if it might be worthwhile to publish a new hex package. There's still plenty of stuff that could be added, I noticed some missing fields on some resources, and there's some deprecation issues with updating products due to the inventory fields. Those should probably be removed or at least handled better for updating products.

For now I've just been dumping it to a map and removing the offending keys so shopify accepts it

@Ninigi
Copy link
Collaborator

Ninigi commented May 29, 2020

@ProtoJazz @balexand I am a maintainer in this project, and I have to apologize for the lack of attention we have been spending on this library.

Truth is (only speaking for myself here, @nsweeting might have a different oppinion) that the initial approach of having structs for each resource has lost a lot of viability since Shopify started to update their API almost every 4 months.
You would also have to keep updating a library of older structs, that are still compatible with older versions, making the library kind of bulky and hard to understand for its use cases.

Ever since Shopify started using versioned REST API, we have largely switched to using their GraphQL API, and I am seeing the same problems there: If you want to use any Shopify API, you better be ready to fully commit to being a Shopify developer, because they are cranking out new features and deprecations in a pace that I find disturbing. My personal opinion is that Shopify probably thinks of Shopify-App developers as basically their employees, and we have to keep up with what they dictate, but I think they are just violating the basic idea of what an API is: A lasting contract between consumer and supplier.

I would like to contribute more, but I simply can not. There is no time management in the world that could give me the freedom to give you guys the attention you deserve.

I spent days basically going through Shopify's REST API documentation and implementing the missing resources, trying to accommodate for their ideas in how a REST API works, coming up with half backed solutions to how we could solve it. I learned a lot, it was my entry into Elixir open-source and I would not want to miss it - but I certainly do not want (or can afford) to spend the rest of my developer life chasing Shopify's new inventions.

@nsweeting by all means, I think it is time to pass the torch, find other maintainers or maybe transfer the whole project.

@ProtoJazz
Copy link

ProtoJazz commented May 29, 2020

Yeah, I suspect as shopify grows they're finding certain parts of their API just not sustainable at scale. They just grow at such a crazy pace.

Thankfully after the change to the versioned api, at least theres some warning about changes and time to fix it

I wouldn't be opposed to helping as a maintainer. I'm still pretty new to elixir, but I've been working with shopify everyday for the last few years and don't see any signs of stopping any time soon

Comment on lines +6 to +7
@enforce_keys [:data, :func, :middleware, :params, :session]
defstruct @enforce_keys
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to @enforce_keys if the enforced keys are the struct keys?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@enforce_keys makes it so that it is an error to instantiate the struct without all of the fields. It has the potential to prevent bugs where a field is accidentally omitted.

image

It's optional and I can remove it if you'd like.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I like the idea.

lib/shopify/enumerable.ex Outdated Show resolved Hide resolved
end
end

def reduce(%Shopify.Enumerable{data: [h | t]} = e, {:cont, acc}, fun) do
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't use h t for head and tail. what is the head? I literally can not even give advice on what it should be, because I don't know what h is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll rename these variables to head and tail. Heads up that pattern matching using the names [h | t] and [head | tail] is a common pattern in Elixir. You can find many examples of both naming conventions in the Elixir source code, like https://github.com/elixir-lang/elixir/blob/b9209ad1ac9c879d7ea17c635a14b22aebbf2242/lib/elixir/lib/enum.ex#L2126-L2128

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

h and t are pretty standard. I dont think naming them head and tail gives any more context on what the data actually is

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are both right, and I am sorry for being unclear. I am talking from a code reading point of view. Sure, I know h and t but what the hell is it in this case!

I still hate on t and h even if there is no way to describe them further, you know why? try to select all t and h in your file and change them to tail and head - you know what I am talking about.

@balexand
Copy link
Contributor Author

Hi @Ninigi. Thanks for your response.

I can definitely relate to your difficulties with the Shopify API. It has lots of quirks and inconsistencies and they do make breaking changes at an alarming rate. I work with the Shopify API constantly at my current job so I really hope things stabilize.

I would prefer to become a regular contributor to this project as opposed to completely taking it over. This could mean giving me access to merge pull-requests in the Github repo. Or it could mean that you would recognize me as a trusted contributor and be willing to merge my pull-requests with less review. In either case, I would consult with the other project members before making any significant changes.

@Ninigi and @nsweeting, would you be interested in having me become a more regular contributor? Also, I'm curious to know how involved you both are interested in being with this project. We can discuss here or I can do a Zoom call if that works better for you.

@balexand
Copy link
Contributor Author

What do you think of removing the middleware option? It complicates the API a bit and I've since moved towards using a custom adapter to throttle requests in my app. Here's the description of the middleware option:

* `:middleware` - An optional middleware function whose primary purpose is to allow an
application to throttle requests to avoid going over Shopify's rate limit. Must be a
function with an arity of 1 that will be passed another function. The middleware must call
that function and return the value returned by that function, which will be `{:ok,
%Shopify.Response{}}` or `{:error, %Shopify.Error{}}`. Defaults to `& &1.()`.

fun
)

{:error, %Error{} = error} ->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is a 3rd case that needs to be considered:

{:error, %Response{}} e.g. Shopify fails (as the API does this relatively often) with a 500. Perhaps a retry in this case would be the best thing to do as paginating over a large collection of items can abort with high cost to re-iterate the whole collection.

Copy link
Contributor Author

@balexand balexand Jun 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I'll look into this. I have a busy week ahead of me so there's a chance I won't get to it until next Monday.

@hashim-sohail
Copy link

@balexand @ProtoJazz
Can't find the documentation to use the change. Can you help me?

@ProtoJazz
Copy link

@balexand @ProtoJazz
Can't find the documentation to use the change. Can you help me?

It hasn't been merged in. You could point at the fork this PR is based on in your mix file https://hexdocs.pm/mix/Mix.Tasks.Deps.html

@hashim-sohail
Copy link

hashim-sohail commented Jun 24, 2020

@balexand @ProtoJazz
Can't find the documentation to use the change. Can you help me?

It hasn't been merged in. You could point at the fork this PR is based on in your mix file https://hexdocs.pm/mix/Mix.Tasks.Deps.html

I have pointed to it, was looking for docs in how to use the new pagination

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Shopify's new cursor-based pagination
5 participants