Using corepack behind corporate proxy and injected certificate

Hi,
I'm willing to use API Platform (https://api-platform.com/) and the Dockerfile of the pwa component requires corepack:

RUN corepack enable && \
	corepack prepare --activate pnpm@latest && \
	pnpm config -g set store-dir /.pnpm-store

The base image is node:20-alpine.

I'm behind a corporate proxy, so I've added the following before using corepack:

RUN set HTTP_PROXY=http://proxy.etat.lu:8080
RUN set HTTPS_PROXY=http://proxy.etat.lu:8080
RUN set ALL_PROXY=http://proxy.etat.lu:8080

We also have a "traffic interceptor" that watches the whole traffic, so I added its certificate (which is already injected in the image):
RUN set NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/etat.crt

However, I'm still running into issues and receive the error below upon executing corepack prepare --activate pnpm@latest:

Internal Error: Error when performing the request to https://registry.npmjs.org/pnpm; for troubleshooting help, see https://github.com/nodejs/corepack#troubleshooting
    at ClientRequest.<anonymous> (/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:42195:14)
    at ClientRequest.emit (node:events:518:28)
    at TLSSocket.socketErrorListener (node:_http_client:495:9)
    at TLSSocket.emit (node:events:518:28)
    at emitErrorNT (node:internal/streams/destroy:169:8)
    at emitErrorCloseNT (node:internal/streams/destroy:128:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
The command '/bin/sh -c corepack prepare --activate pnpm@latest' returned a non-zero code: 1
ERROR: Service 'pwa' failed to build : Build failed

Anybody could help me figure out what I'm missing here?

Thanks in advance.
Adrien.

Probably related to nodejs/undici#2200.

Probably related to nodejs/undici#2200.

To overcome this, should I try RUN set NODE_TLS_REJECT_UNAUTHORIZED=0?
If this is the case, it didn't work :(

Could there be that you need to specify a COREPACK_NPM_REGISTRY in your environment?

Still no luck, even with COREPACK_NPM_REGISTRY.
However, there is something weird. Because when I set this env variable to a fake value, like RUN set COREPACK_NPM_REGISTRY=https://www.microsoft.com/ (even between quotes), I still see that it does the request towards registry.npmjs.org:
Internal Error: Error when performing the request to https://registry.npmjs.org/pnpm; for troubleshooting help, see https://github.com/nodejs/corepack#troubleshooting

Seeing the same issue here with corepack 0.25.x
NODE_EXTRA_CA_CERTS and HTTP_PROXY env variables work as expected in 0.24.x

@merceyz

I've been digging a little more into this issue as I also use a proxy (squid-cache, no proxy auth, no tls inspection).

First, I don't believe this is an injected certificate issue since my proxy only relays connections out without any TLS inspection.

Second, since I fully control my local DNS server and the proxy, I have noticed no connection attempt is made to connect to the defined proxy. However, inspecting the agent object (https://github.com/nodejs/corepack/blob/v0.25.2/sources/httpUtils.ts#L11), it appears to be valid. So the error is fetch() is used (https://github.com/nodejs/corepack/blob/v0.25.2/sources/httpUtils.ts#L15)

Rolling backward, I tried to find what were the changes introduced between 0.24.1 (last known working version for me) and 0.25.0 (first version with that issue). Doing a git blame and I found the PR #365 introduced significant changes in how external data is fetched. Previously https.get(...)was used, but now it's globalThis.fetch(...). Either somehow globalThis.fetch() is broken, or the proxy agent isn't set as expected which causes the internal error.

Last, I found that downgrading to corepack 0.24.1 (npm install -g corepack@0.24.1), then running yarn --version will populate ~/.cache/node/corepack. I believe during this process, tags are retrieved and checked, and the latest known version is stored in ~/.cache/node/corepack/lastKnownGood.json

Then upgrading back to 0.25.2 and running the same command prompts the user to download and install yarn again. Answering yes leads to the same internal error as I described above. So it seems like using globalThis.fetch() breaks more corepack functionalities.

Tagging both @merceyz and @aduh95 who have authored the PR mentioned above for visibility.

When dumped, my ProxyAgent looked like this.

ProxyAgent {
  _events: [Object: null prototype] {},
  _eventsCount: 0,
  _maxListeners: undefined,
  [Symbol(shapeMode)]: false,
  [Symbol(kCapture)]: false,
  [Symbol(destroyed)]: false,
  [Symbol(onDestroyed)]: null,
  [Symbol(closed)]: false,
  [Symbol(onClosed)]: [],
  [Symbol(proxy agent options)]: { uri: 'http://proxy.local-network.net:3129', protocol: 'https' },
  [Symbol(proxy agent)]: Agent {
    _events: [Object: null prototype] {},
    _eventsCount: 0,
    _maxListeners: undefined,
    [Symbol(shapeMode)]: false,
    [Symbol(kCapture)]: false,
    [Symbol(destroyed)]: false,
    [Symbol(onDestroyed)]: null,
    [Symbol(closed)]: false,
    [Symbol(onClosed)]: [],
    [Symbol(dispatch interceptors)]: [ [Function (anonymous)] ],
    [Symbol(options)]: {
      uri: 'http://proxy.local-network.net:3129',
      connect: [AsyncFunction: connect],
      interceptors: undefined
    },
    [Symbol(maxRedirections)]: 0,
    [Symbol(factory)]: [Function: defaultFactory],
    [Symbol(clients)]: Map(0) {},
    [Symbol(onDrain)]: [Function (anonymous)],
    [Symbol(onConnect)]: [Function (anonymous)],
    [Symbol(onDisconnect)]: [Function (anonymous)],
    [Symbol(onConnectionError)]: [Function (anonymous)]
  },
  [Symbol(dispatch interceptors)]: [],
  [Symbol(request tls settings)]: undefined,
  [Symbol(proxy tls settings)]: undefined,
  [Symbol(proxy headers)]: {},
  [Symbol(connect endpoint function)]: [Function: connect],
  [Symbol(proxy client)]: <ref *1> Pool {
    _events: [Object: null prototype] {},
    _eventsCount: 0,
    _maxListeners: undefined,
    [Symbol(shapeMode)]: false,
    [Symbol(kCapture)]: false,
    [Symbol(destroyed)]: false,
    [Symbol(onDestroyed)]: null,
    [Symbol(closed)]: false,
    [Symbol(onClosed)]: [],
    [Symbol(queue)]: FixedQueue {
      tail: [FixedCircularBuffer],
      head: [FixedCircularBuffer]
    },
    [Symbol(clients)]: [],
    [Symbol(queued)]: 0,
    [Symbol(onDrain)]: [Function: onDrain],
    [Symbol(onConnect)]: [Function (anonymous)],
    [Symbol(onDisconnect)]: [Function (anonymous)],
    [Symbol(onConnectionError)]: [Function (anonymous)],
    [Symbol(stats)]: PoolStats { [Symbol(pool)]: [Circular *1] },
    [Symbol(dispatch interceptors)]: [],
    [Symbol(connections)]: null,
    [Symbol(url)]: URL {
      href: 'http://proxy.local-network.net:3129/',
      origin: 'http://proxy.local-network.net:3129',
      protocol: 'http:',
      username: '',
      password: '',
      host: 'proxy.local-network.net:3129',
      hostname: 'proxy.local-network.net',
      port: '3129',
      pathname: '/',
      search: '',
      searchParams: URLSearchParams {},
      hash: ''
    },
    [Symbol(options)]: {
      connect: [Function: connect],
      allowH2: undefined,
      interceptors: undefined
    },
    [Symbol(factory)]: [Function: defaultFactory]
  }
}

Hoping this will be helpful.

Seeing the same issue here with corepack 0.25.x NODE_EXTRA_CA_CERTS and HTTP_PROXY env variables work as expected in 0.24.x

thank you :)

@aureq

@aureq thanks for the detailed report! Can you also confirm whether you're able to reproduce with a simple fetch call outside of Corepack? If the bug is in the fetch implementation, we can write a bug report to Undici issue tracker so it can be fixed upstream.

@aduh95

@aduh95 I'm trying to, but having some typing issues... Could you provide some guidance or help please?

First, I reused most of the code that's present in CorePack to save some time (the code I mentioned in my previous comment).

From the code below, vscode complains with the following error: Object literal may only specify known properties, and 'dispatcher' does not exist in type 'RequestInit'. when trying to set displatch: agent on L28 which seems to indicate the property isn't present. I'm not sure what I'm missing here.

But also, I'm a little puzzled as to why dispatcher of type Dispatcher would accept agent of type ProxyAgent` according to my code below, originally taken from https://github.com/nodejs/corepack/blob/v0.25.2/sources/httpUtils.ts#L17.

Any suggestions welcome.

async function getProxyAgent(input: string | URL) {
    const {getProxyForUrl} = await import(`proxy-from-env`);

    // @ts-expect-error - The internal implementation is compatible with a WHATWG URL instance
    const proxy = getProxyForUrl(input);

    if (!proxy) return undefined;

    // Doing a deep import here since undici isn't tree-shakeable
    const {default: ProxyAgent} = (await import(
      // @ts-expect-error No types for this specific file
      `undici/lib/proxy-agent.js`
    )) as { default: typeof import('undici').ProxyAgent };

    return new ProxyAgent(proxy);
}

async function myFetch(input: string | URL, init?: RequestInit) {
    const agent = await getProxyAgent(input);

    console.error(agent);

    let response;

    try {
        response = await globalThis.fetch(input, {
            ...init,
            dispatcher: agent, // <-- error
        });

    } catch (error) {
        console.error(error);
        throw new Error(`Error when performing the request to ${input}`);
    }
}

fetch("https://checkip.amazonaws.com");

{
    "name": "gh-417",
    "main": "index.ts",
    "engines": {
        "node": "^18.17.1 || >=20.10.0"
    },
    "devDependencies": {
        "@types/node": "^20.10.0",
        "@types/proxy-from-env": "^1",
        "proxy-from-env": "^1.1.0",
        "undici": "^6.6.1",
        "typescript": "^5.0.4",
        "ts-node": "^10.0.0"
    }
}

But also, I'm a little puzzled as to why dispatcher of type Dispatcher would accept agent of type ProxyAgent` according to my code below

As documented in https://undici.nodejs.org/#/docs/api/ProxyAgent, ProxyAgent extends Dispatcher.

From the code below, vscode complains with the following error: Object literal may only specify known properties, and 'dispatcher' does not exist in type 'RequestInit'. when trying to set displatch: agent on L28 which seems to indicate the property isn't present. I'm not sure what I'm missing here.

That's probably because dispatcher is a Undici-specific option. You can see Undici reads the dispatcher property from the option bag either in https://github.com/nodejs/undici/blob/8dea74494cada360ae9d88a23d55b6296b9016ba/lib/web/fetch/request.js#L82 or in https://github.com/nodejs/undici/blob/8dea74494cada360ae9d88a23d55b6296b9016ba/lib/web/fetch/request.js#L107.
I think it's fine to ignore that error, but if you prefer you can also use setGlobalDispatcher(proxyAgent), but I'd expect you'd get the exact same result.
You could also try to reproduce with JS code (instead of TS), to reduce the scope further.

Does the above code reproduce the error? Could you also try to reproduce with URLs Corepack uses, e.g. https://repo.yarnpkg.com/4.0.0/packages/yarnpkg-cli/bin/yarn.js or https://registry.yarnpkg.com/yarn/latest.

@aduh95

@aduh95 Thanks for the information.

It's probably worth mentioning I'm not a JS/TS developer.

With that said, I've simplified my example and the code shared below works as expected. The connection is made to my proxy and the content appears to be retrieved correctly.

So, in my example, undici appears to be working just fine. The corepack code is a little too complex for me to grasp its nuances.

But FWIW, the issue may be on corepack's side.

import * as proxyFromEnv from "proxy-from-env"
import * as proxyAgent from "undici"


async function getProxyAgent(input: string) {
    const proxy = proxyFromEnv.getProxyForUrl(input);
    if (!proxy) return undefined;
    return new proxyAgent.ProxyAgent(proxy);
}

async function myFetch(input: string, init?: RequestInit) {
    const agent = await getProxyAgent(input);

    const u = new URL(input);

    let response;

    try {
        proxyAgent.fetch
        response = await proxyAgent.fetch(u, {
            dispatcher: agent,
        });

    } catch (error) {
        console.error(error);
        throw new Error(`Error when performing the request to ${input}`);
    }

    console.log(response);
}

const res = myFetch("https://checkip.amazonaws.com");

Hi,

I have the same issue and had a look and have a bit more information

My environment

Node 18.19.1
Corepack 0.25.2

1. Uncovering the real error

The error I got initially did not help much:

Internal Error: Error when performing the request to https://repo.yarnpkg.com/4.1.1/packages/yarnpkg-cli/bin/yarn.js; for troubleshooting help, see https://github.com/nodejs/corepack#troubleshooting
    at fetch (/opt/containerbase/tools/corepack/0.25.2/18.19.1/node_modules/corepack/dist/lib/corepack.cjs:22886:11)
    ...

So I added a console.log to get the original error:

original error TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11731:11)
    at async fetch (/opt/containerbase/tools/corepack/0.25.2/18.19.1/node_modules/corepack/dist/lib/corepack.cjs:22880:16)
    at async fetchUrlStream (/opt/containerbase/tools/corepack/0.25.2/18.19.1/node_modules/corepack/dist/lib/corepack.cjs:22918:20)
    ...
  cause: TypeError: this[kClient].connect is not a function
      at Client.connect (/opt/containerbase/tools/corepack/0.25.2/18.19.1/node_modules/corepack/dist/lib/corepack.cjs:13364:66)
      at socket (/opt/containerbase/tools/corepack/0.25.2/18.19.1/node_modules/corepack/dist/lib/corepack.cjs:12289:29)
      at ...
}

So what we get is TypeError: this[kClient].connect is not a function
apparently, from this line : https://github.com/nodejs/undici/blob/v6.6.2/lib/proxy-agent.js#L93

This error is internal to undici (the bundled library) but is triggered when calling globalThis.fetch

One thing I noticed is that Corepack bundles Undici 6.6.2
but Node.JS bundles version 5.28.3 : https://github.com/nodejs/node/releases/tag/v18.19.1

But from what I found next this seems to have no implication

2. A short reproduction:

{
    "devDependencies": {
        "proxy-from-env": "^1.1.0",
        "undici": "6.6.2"
    }
}

import { getProxyForUrl } from "proxy-from-env";
import ProxyAgent from "undici/lib/proxy-agent.js";
//import { fetch } from "undici/lib/fetch/index.js";
//import { ProxyAgent, fetch } from "undici";

async function getProxyAgent(input) {
    const proxy = getProxyForUrl(input);
    if (!proxy) return undefined;
    return new ProxyAgent(proxy);
}

async function myFetch(input, init) {
    const agent = await getProxyAgent(input);

    try {
        const response = await globalThis.fetch(input, {
            dispatcher: agent,
        });
        console.log(response.status);
    } catch (error) {
        console.error(error);
        throw new Error(`Error when performing the request to ${input}`);
    }
}

myFetch("https://repo.yarnpkg.com/4.1.1/packages/yarnpkg-cli/bin/yarn.js");

Using this with a proxy will yield the error we saw above:

TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11731:11)
    at async myFetch (file:///home/sgoetz/workspace/sq/repro/fetch.mjs:18:20) {
  cause: TypeError: this[kClient].connect is not a function

But by changing the following import, it works OK :

- import ProxyAgent from "undici/lib/proxy-agent.js";
+ import { ProxyAgent } from "undici";

My understanding is that some other file imported by undici has side effects and would get the library to work fine.

This really needs to be fixed ... we do have the exact same error ...

@boindil

@boindil As suggested, if this is blocking you, you have 2 options.

Downgrade to 0.24.x as suggested by me
Patch the source as suggested by @onigoetz

And if this is a pressing issue, then feel free to submit a pull request for the team to review.

I have the same issue:

Internal Error: Error when performing the request to https://registry.npmjs.org/pnpm; for troubleshooting help, see https://github.com/nodejs/corepack#troubleshooting
    at ClientRequest.<anonymous> (C:\Program Files\nodejs\node_modules\corepack\dist\lib\corepack.cjs:42195:14)

But my corepack version is 0.23.0, so downgrade to 24 is not possible, I have an even lower version.

I Upgraded a project from Node 16 to 20. Our build is made in a custom Docker image. I think that the bugged version of corepack is shipped by default in Node 20. I ended up upgradind corepack to 0.27.0

For reference:

RUN corepack enable && npm install -g corepack@0.27.0 && corepack install --global yarn@1.22.22

Works well behind our proxy.

@devbydixon as stated above, you need to upgrade your corepack to 0.27.0 or above. If you're still having a problem, please open a new issue as this one is now closed.

Navigation Menu

Repository navigation

Using corepack behind corporate proxy and injected certificate #417

My environment

1. Uncovering the real error

2. A short reproduction:

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Using corepack behind corporate proxy and injected certificate #417

Description

Activity

merceyz commented on Mar 6, 2024

adurand-ctie commented on Mar 6, 2024

aduh95 commented on Mar 6, 2024

adurand-ctie commented on Mar 6, 2024

jlaminate commented on Mar 7, 2024

aureq commented on Mar 29, 2024

marwin1991 commented on Apr 2, 2024

aduh95 commented on Apr 2, 2024

aureq commented on Apr 2, 2024

aduh95 commented on Apr 2, 2024

aureq commented on Apr 3, 2024

onigoetz commented on Apr 3, 2024

My environment

1. Uncovering the real error

2. A short reproduction:

boindil commented on Apr 4, 2024

aureq commented on Apr 4, 2024

twojnarowski commented on Apr 18, 2024

jsayer101 commented on Apr 30, 2024

aureq commented on Jun 14, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions