Ok it looks like this started happening in https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/SmashPig/+/1222300/7/PaymentProviders/Gravy/Mapper/ResponseMapper.php, and it was then partially fixed in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/1260070.

I see we had an older ticket for this T420651: PayPal recurring fail wellness check

However, it looks like we still need to clean up the monthly convert flow, which is still using the processor_contact_id here

			foreach ( [ 'processor_contact_id', 'fiscal_number' ] as $optionalKey ) {
				if ( !empty( $sessionData[$optionalKey] ) ) {
					$message[$optionalKey] = $sessionData[$optionalKey];
				}
			}

We were clean them directly from the db for paypal recurring, I updated the processor_contact_id to NULL, validate with below should see nothing.

MariaDB [civicrm]> SELECT
    -> cr.id,
    -> cr.create_date,
    -> pp.name AS processor_name,
    -> crs.processor_contact_id,
    -> crs.id as delete_id
    ->     FROM
    -> civicrm_contribution_recur cr
    -> JOIN civicrm_contribution_recur_smashpig crs ON crs.entity_id = cr.id
    -> JOIN civicrm_payment_processor pp ON pp.id = cr.payment_processor_id
    ->     WHERE
    -> cr.payment_processor_id = 19
    -> AND crs.processor_contact_id IS NOT NULL
    ->       AND processor_contact_id NOT REGEXP '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$'
    -> ORDER BY
    -> cr.id DESC;

Change #1276835 had a related patch set uploaded (by Wfan; author: Wfan):

[mediawiki/extensions/DonationInterface@master] Unset processor_contact_id if paypal

https://gerrit.wikimedia.org/r/1276835

@AnnWF While this is the same error we saw with the paypal ones I think something else is going on here. Recurring convert isn't turned on for gravy paypal.

I dug into one of the errors from yesterday:
cid=56200564
Its a adyen credit card recurring but failed with the same reason
2026-04-23 06:56:24,626 ERROR civicrm.wmf.INFO: Charging contribution_recur id: 2764507 with invoice_id: 245656263.2
2026-04-23 06:56:24,834 ERROR civicrm.wmf.ALERT: Validation error during recurring charge, in field: processor_contact_id. Message: Input should be a valid UUID, invalid character: expected an optional prefix of urn:uuid: followed by [0-9a-fA-F-], found . at 10. Payment Method: cc. Payment Submethod: . Request: {"recurring_payment_token":"710f184c-c932-490a-a54e-d3baa7a2920e","amount":5.35,"currency":"EUR","first_name":"*","last_name":"","email":"","country":"AT","order_id":"245656263.2","installment":"recurring","description":"Wikimedia 877 600 9454","recurring":true,"user_ip":"*","processor_contact_id":"245656263.1","fiscal_number":null}. Response: {"type":"error","code":"bad_request","status":400,"message":"Request failed validation","details":[{"location":"body","pointer":"/buyer_id","message":"Input should be a valid UUID, invalid character: expected an optional prefix of urn:uuid: followed by [0-9a-fA-F-], found . at 10","type":"uuid_parsing"}]}
2026-04-23 06:56:24,835 ERROR civicrm.wmf.INFO: Error: 1000014 invoice_id:245656263.2
2026-04-23 06:56:24,835 ERROR civicrm.wmf.INFO: Raw response: {"type":"error","code":"bad_request","status":400,"message":"Request failed validation","details":[{"location":"body","pointer":"\/buyer_id","message":"Input should be a valid UUID, invalid character: expected an optional prefix of urn:uuid: followed by [0-9a-fA-F-], found . at 10","type":"uuid_parsing"}]}

I tried looking in the gravy console for more info but I can't find the attempt in there, I asked on the slack channel if they could find anything

Ah okay so it wont be in the gravy console because of that validation error. I think we maybe just need to stop sending the processor_contact_id for all recurring converts

Added regex check before send over processor_contact_id, this should catch the above issues

Change #1276835 merged by jenkins-bot:

[mediawiki/extensions/DonationInterface@master] Unset processor_contact_id if not uuid

https://gerrit.wikimedia.org/r/1276835

Change #1277613 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/SmashPig@master] Map paypal payer ID to backendProcessorContactID

https://gerrit.wikimedia.org/r/1277613

Change #1277615 had a related patch set uploaded (by Ejegg; author: Ejegg):

[mediawiki/extensions/DonationInterface@master] Revert "Unset processor_contact_id if not uuid"

https://gerrit.wikimedia.org/r/1277615

Change #1277616 had a related patch set uploaded (by Ejegg; author: Ejegg):

[mediawiki/extensions/DonationInterface@master] More targetted processor_contact_id filter

https://gerrit.wikimedia.org/r/1277616

Change #1277615 merged by jenkins-bot:

[mediawiki/extensions/DonationInterface@master] Revert "Unset processor_contact_id if not uuid"

https://gerrit.wikimedia.org/r/1277615

Change #1277616 merged by jenkins-bot:

[mediawiki/extensions/DonationInterface@master] More targetted processor_contact_id filter

https://gerrit.wikimedia.org/r/1277616

Ok, I found that original batch I showed and pulled out their backend processor. It's a mix of PayPal and Adyen.

MariaDB [civicrm]> SELECT 
    ->   cr.id, 
    ->   crs.processor_contact_id, 
    ->   x.backend_processor, 
    ->   COUNT(c.id) AS contribution_count 
    -> FROM 
    ->   civicrm_contribution_recur cr 
    ->   JOIN civicrm_contribution_recur_smashpig crs ON crs.entity_id = cr.id 
    ->   JOIN civicrm_contribution c ON c.contribution_recur_id = cr.id 
    ->   JOIN wmf_contribution_extra x ON x.entity_id = c.id 
    -> WHERE 
    ->   cr.id IN (
    ->     2778114, 2777222, 2775224, 2774761, 
    ->     2774760, 2773291, 2773094, 2772522, 
    ->     2772326, 2772273, 2769945, 2768592, 
    ->     2768456, 2768418, 2768331, 2768125, 
    ->     2767503, 2766962, 2766645, 2766594
    ->   ) 
    -> GROUP BY 
    ->   cr.id, 
    ->   crs.processor_contact_id, 
    ->   x.backend_processor;
+---------+----------------------+-------------------+--------------------+
| id      | processor_contact_id | backend_processor | contribution_count |
+---------+----------------------+-------------------+--------------------+
| 2766594 | NULL                 | adyen             |                  1 |
| 2766645 | NULL                 | paypal            |                  1 |
| 2766962 | NULL                 | paypal            |                  1 |
| 2767503 | NULL                 | paypal            |                  1 |
| 2768125 | NULL                 | paypal            |                  1 |
| 2768331 | NULL                 | paypal            |                  1 |
| 2768418 | NULL                 | paypal            |                  1 |
| 2768456 | NULL                 | adyen             |                  1 |
| 2768592 | NULL                 | adyen             |                  1 |
| 2769945 | NULL                 | adyen             |                  1 |
| 2772273 | NULL                 | paypal            |                  1 |
| 2772326 | NULL                 | paypal            |                  1 |
| 2772522 | NULL                 | paypal            |                  1 |
| 2773094 | NULL                 | paypal            |                  1 |
| 2773291 | NULL                 | paypal            |                  1 |
| 2774760 | NULL                 | paypal            |                  1 |
| 2774761 | NULL                 | paypal            |                  1 |
| 2775224 | NULL                 | paypal            |                  1 |
| 2777222 | NULL                 | paypal            |                  1 |
| 2778114 | NULL                 | paypal            |                  1 |
+---------+----------------------+-------------------+--------------------+
20 rows in set (0.003 sec)

Yeah the paypal one is only payerID, and adyen one is orderID, thus I used the UUID type check after Christine points out this

but we do have this https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/1277616 to catch it I believe this should be gone, we are not limited this check to paypal

Yeah, that should fix them all.

Change #1277613 merged by jenkins-bot:

[wikimedia/fundraising/SmashPig@master] Map paypal payer ID to backendProcessorContactID

https://gerrit.wikimedia.org/r/1277613