How to handle IPN failure behavior?

[PseudoNyhm]

I've been able to reproduce a frustrating failure case, but not sure if there's anything s2Member can do about it.

Scenario: A level 0 user clicks PayPal button and completes PayPal transaction for a recurring (subscription) account. PayPal redirects back to my site. However, the IPN message has not yet come through (because I blocked it). Result: The user is still a Subscriber (not Level 1) and has no "Paid Subscr. ID".

Conclusion: If the IPN message fails/delays, my s2Member site still thinks the user is a non-paid subscriber (level 0). I think IPN retries later, but there is a gap when the user is frustrated and cannot get Level 1 access (and may even buy another subscription thinking it was their fault).

Questions: I have PDT configured. Can s2Member use this data to set the user to Level 1 immediately upon return? Also, can I write a page that includes a success/fail message based on the PayPal PDT results?

[Cristián Lávaque]

Well, s2Member depends on what PayPal tells it about the user. What do you mean the website blocked the IPN message?

Could you post, x'ing out any private info, the entries for this transaction in the s2Member PayPal logs? /wp-content/plugins/s2member-logs/

[PseudoNyhm]

I'm really pleased to see that s2Member keeps such good logs. I'm protocol-savvy, but this is my first look at PayPal transactions. Please feel free to correct me if I misinterpret something, but I see exactly what happened.

Success case: IPN signals a 'txn_type' => 'subscr_signup', then later a 'txn_type' => 'subscr_payment' and everything is good.

Fail case: At the time the IPN signup signal was sent, something blocked the message (eg, PayPal IPN was lagging, Internet routing issues, or my site went down). Later, an IPN payment signal came in for an unknown subscr_id:

Code: Select all

array (
    0 => 'IPN received on: Sun Jun 26, 2011 10:57:09 am UTC',
    1 => 's2Member POST vars verified through a POST back to PayPal®.',
    2 => 's2Member originating domain ( _SERVER[HTTP_HOST] ) validated.',
    3 => 's2Member txn_type identified as subscr_payment|recurring_payment.',
    4 => 'Sleeping for 5 seconds. Waiting for a possible subscr_signup|subscr_modify|recurring_payment_profile_created.',
    5 => 'Awake. It\'s Sun Jun 26, 2011 10:57:14 am UTC. s2Member txn_type identified as subscr_payment|recurring_payment.',
    6 => 'Skipping this IPN response, for now. The Subscr. ID is not associated with a registered Member.',
    7 => 'Re-generating. This IPN will go into a Transient Queue; and be re-processed during registration.',
  ),


I notice that s2Member will wait for the user to register to process the subscr_id, but that's not valid in this case (the user is already registered, but s2Member missed the initial signup IPN). Notice that this is not s2Member's fault (technical issues caused the IPN to not go through), but it seems that s2Member cannot recover.

The end result is that I have a registered user who has paid, but is still at Level 0. PayPal is taking their money, and sending IPN payment signals to s2Member, but s2Member doesn't know who to assign the credit to. In the meantime, the user can't use my site.

(Worse, the user may decide to subscribe again, which results in multiple recurring payments for the same user. I talked to PayPal about this, and there is no way to prevent users from subscribing more than once.)

Can s2Member recover from this failure case?

[PseudoNyhm]

I tried a few things to correct the situation of s2Member not knowing that my test user had paid:

1. This doesn't work: Tell s2Member to regenerate Registration Access Link (I found the subscr_id of the test user in the ipn log). This was not right because the user was already registered (the link just returned them to the registration page).
2. Possible work-around: Manually enter subscr_id (found in subsequent IPN messages that had no associated member) and other fields into WordPress user account screen. Then would have to set user to Level 1. (More testing needed to see if future IPN payment signals would actually be associated with user.)
3. Partial work-around: Use PayPal IPN history to manually identify and resend the lost IPN message. This seemed to work (but further testing needed).

Can you advise me what is the best way to administratively handle this situation if/when it arises in practice?

Also, would it be possible for s2Member to instantly upgrade the user when PDT returns success (without having to rely on the initial IPN, which could be delayed)?

Please let me know if there's any other scenario you'd like me to report on or run, while I have it set up.

[Cristián Lávaque]

I'll email Jason asking him about it. Thanks a lot for the great feedback and testing.

By the way, you can manually edit the user's profile to give him the access they he paid for WP Admin -> Users -> Edit Profile. I know, though, that's not what worries you, but avoiding the potential trouble that may come from this situation.

[Jason Caldwell]

Thanks for the excellent question.
Yes, s2Member is capable of handling things in real-time with PayPal's PDT service. So if you setup Auto-Return/PDT, and also setup IPN, this is your best bet. When/if both of these services fail, I would suggest editing the User's account manually after doing a review of the transaction. That's about all we can do in this case. IPN is really intended to be used as the final word from PayPal. If both PDT and IPN fail, as you said, there's not much s2Member can do, except to make it possible for you to edit the account manually inside your Dashboard, so that unforeseen circumstances can be rectified by the site owner.

[PseudoNyhm]

Thank you for your advice. That is a reasonable work-around (that is, to manually augment the account and review the situation). Also thank you for clarifying that PDT is intended to provide preliminary approval of subscription (and s2Member should set user to Level 1 upon PDT success result).

However, my failure case is that only IPN was blocked. The PDT return was successful, but the user was not assigned Level 1 (or a subscr_id) until I forced PayPal to resend the IPN. I will be setting up another test sometime this week, and will report back.

[PseudoNyhm]

Finally I get some time to test s2Member some more... Here's what my paypal-rtn.log file shows when I create a PayPal subscription. Notice that everything appears to go as planned from the user's point-of-view (click paypal button, log into paypal, click Agree button, get forwarded back to my test site). However, upon return I show up t the home page (as logged), and I'm still at Level 0 (because my users register prior to "upgrading" to Level 1 with a PayPal subscription modification button).

Notes:

1. For this test case, IPN messages are being delayed/lost on purpose. The purpose is to make sure the PDT return is being processed. Why doesn't PayPal include proper PDT data? Have I missed something obvious?
2. I am using a non-encrypted button (that's another issue to test today...)
3. I have confirmed that the button code includes the proper return (as well as my paypal seller account settings)
4. EDIT: This same message is logged even when IPN are NOT blocked. So, in all cases, I'm getting no PDT data from PayPal (Sandbox)

paypal-rtn.log

Code: Select all

WordPress® v3.1.4 :: s2Member® v110620
xxxxxxxxxxx.com/?s2member_paypal_return=1
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.18) Gecko/20110614 Firefox/3.6.18
array (
  's2member_log' =>
  array (
    0 => 'No Return-Data from PayPal®. Customer must wait for Email Confirmation.',
    1 => 'Redirecting Customer to the Home Page.',
  ),
)


Thank you for any direction you can offer.

[Jason Caldwell]

Are you offering a free trial period?
If so, please see this thread for details:
viewtopic.php?f=4&t=521&p=2849&hilit=PDT+trial+immediately#p2849
PayPal does not send s2Member any PDT data when/if there is a free trial period.

Please post a copy of your Shortcode if you continue to have trouble. Or, if you're using a RAW Button Code, please post that as well so we can see. With RAW Button codes, please make sure that rm="2". The Shortcode handles this for you automatically, but if you've designed your own Button Code, you might want to have a look here: https://www.x.com/docs/DOC-1552

[PseudoNyhm]

A-ha! That's it exactly; I'm specifying a free trial period. If this caveat was in the docs anywhere, I missed that point. Thank you for the clarification.

Notice that my users are already registered (Level 0) when buying a subscription. To handle my case, I could change the button code to return to a custom page, rather than the proper s2member_paypal_return path. The custom page would check the user's role to see if they are Level 1 yet, and if not explain that the upgrade process can take up to N hours, or some such.

By circumventing the s2member_paypal_return processing, am I losing important s2Member tracking/logging? Is there a way to specify the proper s2member_paypal_return path, but also redirect to my custom return page (if/when there is no PDT data)?

Or, is there a better way you'd advise handling this PayPal deficiency? (Not offering a free trial or using PayPal Pro are not suitable for my needs.)

In your experience with PayPal, am I just over-preparing for a fringe failure case (in which IPNs are delayed/lost)? How often does this happen in practice? If the IPN comes in before the PayPal redirect, all is well (except that the return page is still the home page).

Thank you very much for your excellent support.

[Jason Caldwell]

By circumventing the s2member_paypal_return processing, am I losing important s2Member tracking/logging? Is there a way to specify the proper s2member_paypal_return path, but also redirect to my custom return page (if/when there is no PDT data)?

It's perfectly OK to do this. All Payment Gateways integrated with s2Member include Auto-Return/PDT handlers ( or some variation thereof, depending on the Payment Gateway being integrated ); however, these PDT integrations are ony designed to allow instant gratification when/if possible; they are NOT required for s2Member's processing to succeed. In other words, it's fine to bypass /?s2member_paypal_return=1, if you prefer.

In fact, with s2Member Pro installed, a new Shortcode Attribute is made available ( success="http://your/custom/thank-you-page" ) which makes this easy to accomplish.

It's NOT OK to bypass IPN processing though. s2Member uses IPN communication ( or a variation thereof, depending on the Payment Gateway being integrated ); which is always the best ( i.e. most reliable ) method of processing a transaction after checkout.

In practice, IPN communication is VERY reliable. Particularly with PayPal. IPN communication occurs almost instantly. Of course, there are exceptions to this, but in my experience those occurrences are VERY rare, and when/if a delay does occur, it's usually with IPNs related to cancellation, expiration, refunds, etc... and usually NOT with sales transactions, like the ones you're testing with.

[PseudoNyhm]

Excellent! Thank you for this precise answer. I will likely experiment with a custom return page, which can check for the level assignment or provide helpful advice if the IPN has not yet arrived. Unfortunately, PayPal does not appear to offer an alternate "customer has declined/canceled" return page, so I won't know if they chose not to subscribe versus IPN delay.

In any case, I think I have a good understanding of the workflow now.

[Jason Caldwell]

Excellent. I'm glad we could help.
~ thanks for reporting back!

[Cristián Lávaque]

I just wanted to add a comment on ClickBank which, unlike the other gateways, doesn't have the success attribute for the shortcode, and the return URL does have to be /?s2member_pro_clickbank_return=1 or the person won't have the right cookies set by s2Member to be allowed registration. So you have to use that return URL, but you can still customize the thank-you page forwarding the person there using this hack Jason sent me once:

/wp-content/mu-plugins/s2hacks.php

Code: Select all

<?php
add_action ('ws_plugin__s2member_during_paypal_return_during_subscr_signup_wo_update_vars', 'my_clickbank_return');
function my_clickbank_return () {
    if ($_REQUEST['s2member_paypal_proxy'] === 'clickbank') {
        echo '<script type="text/javascript">', "\n", 
             "window.location = 'http://www.example.com/thank-you/';", "\n", 
             '</script>', "\n";
        exit ();
        // Note. Don't use wp_redirect() or header("Location: xxx") here.
        // The cookies set by s2Member may not be read properly.
        // Always use window.location as shown above.
    }
}
?>

[Jason Caldwell]

Awesome. And Cristián, just so you know, s2Member stills falls back on the ClickBank IPN handler. So you're correct, a Customer would need to be redirected somewhere after checkout. Using s2Member's built-in handler is recommended. However, s2Member's IPN handler will always process the ClickBank transaction behind the scene, and send the Customer an email. The link in that email serves as a fallback, in case something out of the ordinary happens in real-time on your site. For example, if you use a custom Thank-You Page URL for ClickBank integration, the email that s2Member processes ( customizable in your Dashboard ), will still be sent, regardless. Or, another example would be if a Customer loses their Internet connection while being returned to your site, or they get lost somehow.

For the benefit of other readers:
All of that being said. Please DO pay attention to what Cristián pointed out above. If you're going to use a Custom Thank-You URL with ClickBank, or any of s2Member's Gateway integrations, please make sure that you know what you're doing. In other words, one size does NOT fit all when it comes to a Thank-You Page URL. s2Member handles Thank-You Page URLs dynamically, based on what the Customer is doing ( i.e. signing up for the first time, modifying an existing account, etc ).

At this time ( i.e. s2Member v110620 ), the only Payment Gateway integrations that actually provide support for the success="" attribute in your Shortcode ( which is perfectly safe to use ), are the following:

- PayPal Standard Buttons ( you can use success="http://.. " in your Shortcode )
- PayPal Pro Forms ( you can use success="http://.. " in your Shortcode )
- Authorize.Net Pro Forms ( you can use success="http://.. " in your Shortcode )

In all cases, this requires s2Member Pro. The success="" attribute is available ONLY with s2Member Pro.

[Cristián Lávaque]

Great! Thanks for explaining that.

[PseudoNyhm]

I've now tested the other side of the IPN failure coin: If the subscription cancellation IPN signal is lost, s2Member never notices and the user remains at Level 1 (forever?).

In this case, does s2Member auto-degrade the user upon expected EOT (when the next expected recurring payment never shows up)? Sorry, I haven't taken the time to allow this to lapse in my test yet.

If not, is there a way s2Member could do a periodic (monthly) sweep, confirming the status of each active member, just to make sure all is in sync?

If not, what is the impact on s2Member if I were to force PayPal to resend a bunch of IPNs now an then just to make sure all is synced up? (I know that's sloppy account management, but I just want to weigh my options.)

Otherwise, someone would have to review this manually, which would be quite a bit of overhead to match each PayPal seller with the WordPress/s2Member user.

Thanks for clarifying all my nitpicking use cases!

[Jason Caldwell]

I've now tested the other side of the IPN failure coin: If the subscription cancellation IPN signal is lost, s2Member never notices and the user remains at Level 1 (forever?).

In this case, does s2Member auto-degrade the user upon expected EOT (when the next expected recurring payment never shows up)? Sorry, I haven't taken the time to allow this to lapse in my test yet.

No. s2Member depends upon communication from PayPal's IPN system in order to handle the cancellation of an ongoing recurring payment profile. If this were a fixed term, s2Member WOULD know ahead of time when to expire the Member, but in cases where the charges are ongoing, s2Member must rely upon PayPal's IPN system. If IPN communication fails, the Member will remain as they were. In your test case, at Level #1 ( forever ).

If not, is there a way s2Member could do a periodic (monthly) sweep, confirming the status of each active member, just to make sure all is in sync?

Sorry, PayPal Standard integration does not provide API access that would allow such a thing. s2Member's integration with Authorize.Net DOES support this though. However, each Payment Gateway has it's up sides and down sides. Overall, my recommendation is PayPal Standard or PayPal Pro ( ideally ).

If not, what is the impact on s2Member if I were to force PayPal to resend a bunch of IPNs now an then just to make sure all is synced up? (I know that's sloppy account management, but I just want to weigh my options.)

Hmm. That's an interesting idea. I'll take a look to see if we can make this possible in a future release. However, as of s2Member v110620, I would NOT recommend this. s2Member should be fine if duplicate IPNs are regarding cancellations, expirations, refunds, etc. But you definitely do NOT want to send duplicate copies of IPNs pertaining to subscription signups. That would cause havoc with s2Member internally, yielding unexpected results. I'll see what we can do in a future release to improve that.

[Jason Caldwell]

Regarding PayPal Pro.
PayPal Pro integration DOES provide an API that would make a monthly sweep possible.
See: viewtopic.php?f=4&t=2062#p7407

As of s2Member v110620, you would need to implement this yourself though, as s2Member does NOT do this ( i.e. a monthly sweep ) on it's own in the current release.

FYI: You can't run PayPal Standard for awhile, and then switch over to PayPal Pro and expect a monthly sweep to work as intended. The PayPal Pro API will ONLY return a valid status for Recurring Profiles that were originally created through the PayPal Pro API. In other words, if it is your intention to run a monthly sweep, be sure that you start out using s2Member Pro with a PayPal Pro account please.

[PseudoNyhm]

Many thanks for your precise advice. To start out, I plan to stick with PayPal Standard (with the understanding that a periodic user checkup sweep is not possible, and will be difficult if upgrading to PayPal Pro). I had previously resent some IPNs, and s2Member seemed OK with it, but I will avoid this in practice (unless I can identify a specific IPN that is not a signup). If/when these things happen, I'll have to deal with them manually on a case-by-case basis.

Now I know how it behaves ... and knowing is half the battle.

[Jason Caldwell]

Gotchya. Cool.
Thanks for raising these great points.
~ much appreciated!