How a Retired Cuban Currency Made My Laravel Tests Flaky

Most flaky tests I have encountered involved time or shared state. This one was different: it was caused by a currency that had been withdrawn from circulation years earlier.

I was running my Laravel test suite when a test that normally passed suddenly failed. The endpoint was expected to create a product and return 201, but it returned 422 instead:

Expected response status code [201] but received 422.

The price.currency must be 3-letters ISO currency code.

Running the failed test again did not reproduce the problem. Neither did the next several runs. The actual cause was hidden in the generated request payload.

The Unexpected Value

The product payload used Faker to generate its currency:

$currency = fake()->currencyCode();

Most values returned by this method, such as USD, EUR, or GBP, passed validation. Very occasionally, however, Faker returned:

CUC

CUC is the code for the Cuban convertible peso. Cuba began withdrawing it from circulation in 2021, and it is now obsolete. You can read more about its history on Wikipedia.

Faker v1.24.1 still contains CUC in the currency list used by currencyCode(). The relevant list can be found in Faker's Miscellaneous provider.

My validation rule intentionally accepted only the currencies supported by the application, and CUC was not one of them. Both components behaved consistently with their own data sets, but those data sets did not match:

Faker currency list            Application currency list
        CUC          ──X──>           rejected

That mismatch was enough to make the test fail.

Why the Failure Was So Rare

In Faker v1.24.1, currencyCode() randomly selects from 154 codes. Only one of those codes was rejected by my validation list, so the probability of receiving the problematic currency was:

1/154 ≈ 0.65%

With roughly a 0.65% chance on each call to currencyCode(), the failure is rare enough to disappear when rerunning the test, but common enough to surface eventually in a frequently executed suite.

Step 1: Reproduce the Failure Deterministically

The first step was to stop relying on chance. I started with an otherwise valid payload and replaced only the generated currency:

$payload = $this->makeValidProductPayload();
$payload['price']['currency'] = 'CUC';

$response = $this->postJson('/products', $payload);

$response
    ->assertUnprocessable()
    ->assertJsonValidationErrors(['price.currency']);

This confirmed that the 422 response was produced by currency validation rather than storage, authentication, or another part of the request.

I also compared Faker's currency list with the accepted list. CUC was the only value Faker could generate that the validation rule rejected.

Step 2: Choose the Correct Source of Truth

One tempting fix would be to add CUC to the validation rule. That would make the test pass, but it would also change production behavior and allow a currency the application did not intend to support.

The validation rule was correct for the product. The test data generator was the component that needed to follow it.

The simplest local fix would be:

$currency = fake()->randomElement(\App\Rules\CurrencyCode::$currencyCodes);

That works when currencies are generated in only one place. I wanted to preserve the familiar fake()->currencyCode() API and guarantee the same behavior everywhere, so I created a custom Faker provider instead.

Step 3: Override currencyCode() With a Custom Provider

Faker allows custom providers to define new formatters or override existing ones. I added a provider whose currencyCode() method selects directly from the accepted list:

<?php

namespace App\Faker;

use App\Rules\CurrencyCode;
use Faker\Provider\Base;

class CustomFakerProvider extends Base
{
    /**
     * Generates a currency code from a custom list of supported currencies.
     *
     * Faker v1.24.1 includes the obsolete 'CUC' code in its currency list, while
     * some validation rules accept only application supported currencies.
     *
     * Faker's default currencyCode() can therefore cause occasional validation failures
     * and flaky tests. This override keeps generated values within the accepted list.
     *
     * @see https://en.wikipedia.org/wiki/Cuban_convertible_peso
     */
    public function currencyCode(): string
    {
        return static::randomElement(CurrencyCode::$currencyCodes);
    }
}

Because this provider is added after Faker's default providers, its method becomes the formatter used by:

fake()->currencyCode();

Existing call sites do not need to change.

Step 4: Register the Provider Lazily in Laravel

Calling fake() inside a service provider's boot() method would work, but it would instantiate Faker every time the application boots, even when Faker is never used.

Instead, I registered a resolving hook for Faker\Generator:

<?php

namespace App\Providers;

use App\Faker\CustomFakerProvider;
use Faker\Generator;
use Illuminate\Support\ServiceProvider;

class FakerServiceProvider extends ServiceProvider
{
    public function register(): void
    {
        // fakerphp/faker is a development dependency
        // and may not be installed in production,
        // so skip registering the hook when the package is unavailable.
        if (!class_exists(Generator::class)) {
            return;
        }

        $this->app->resolving(Generator::class, function (Generator $faker) {
            $faker->addProvider(new CustomFakerProvider($faker));
        });
    }
}

This approach has three useful properties:

• Faker remains lazy and is created only when requested.
• The hook applies to generators created for any locale.
• Production remains safe when fakerphp/faker is installed only as a development dependency.

Register the service provider in bootstrap/providers.php:

<?php

return [
    App\Providers\AppServiceProvider::class,
    App\Providers\FakerServiceProvider::class,
];

Laravel's fake() helper resolves locale-specific generator instances internally. A resolving hook is useful here because it reacts to the resulting Faker\Generator object rather than depending on a particular locale-specific container key.

Step 5: Verify the Override

A deterministic test can confirm that the custom formatter is registered:

use App\Faker\CustomFakerProvider;

it('uses the custom currency formatter', function () {
    [$provider] = fake()->getFormatter('currencyCode');

    expect($provider)->toBeInstanceOf(CustomFakerProvider::class);
});

This verifies the override directly instead of relying on a random sample of generated values.

After registering the provider, the original test could continue using fake()->currencyCode() without occasionally producing CUC.

The Broader Lesson

Random test data is useful, but every random generator has an implicit domain. Flaky behavior appears when that domain is broader than the domain accepted by the code under test.

The durable fix is not to rerun the failed test or seed the random generator until the failure disappears. It is to make generated data obey the same constraints as production input.

In this case, one retired Cuban currency revealed the mismatch. Once the validation list became the source of truth for Faker as well, the test stopped being flaky without weakening validation or changing existing call sites.