Mike
Mike•9mo ago

Trouble committing file to GitHub but only in bun runtime

I'm trying to create a script to commit font files into my github repository. This is code that is migrated over from airplane.dev. Initially i created the code using the Deno runtime, this works but I want to use fontkit, which does not support the "import ... from ..." imports. So the following code WORKS: 
import * as wmill from "npm:windmill-client@1";
import { Octokit } from "npm:@octokit/rest@20.0.2";

export async function main(
  font_file: wmill.Base64,
  font_name: string,
  font_type: string,
) {
  const octokit = new Octokit({
    auth: (await wmill.getVariable("f/COMPANY/GITHUB_TOKEN")),
  });

  const acceptedFileTypes = ["otf", "ttf"];

  if (acceptedFileTypes.includes(font_type) && font_name && font_name !== "") {
    const result = await octokit.repos.createOrUpdateFileContents({
      owner: "OWNER",
      repo: "REPO",
      path: 'PATH',
      message: 'MESSAGE',
      content: font_file,
    });

    const res = await fetch(
      "ENDPOINT TO NOTIFY FILE IS UPLOAD, NOT RELEVANT FOR THIS ISSUE",
    );

    const responseText = res.text();

    return responseText || 500;
  } else {
    return "error: unsupported file extension, must be otf or ttf.";
  }
}
import * as wmill from "npm:windmill-client@1";
import { Octokit } from "npm:@octokit/rest@20.0.2";

export async function main(
  font_file: wmill.Base64,
  font_name: string,
  font_type: string,
) {
  const octokit = new Octokit({
    auth: (await wmill.getVariable("f/COMPANY/GITHUB_TOKEN")),
  });

  const acceptedFileTypes = ["otf", "ttf"];

  if (acceptedFileTypes.includes(font_type) && font_name && font_name !== "") {
    const result = await octokit.repos.createOrUpdateFileContents({
      owner: "OWNER",
      repo: "REPO",
      path: 'PATH',
      message: 'MESSAGE',
      content: font_file,
    });

    const res = await fetch(
      "ENDPOINT TO NOTIFY FILE IS UPLOAD, NOT RELEVANT FOR THIS ISSUE",
    );

    const responseText = res.text();

    return responseText || 500;
  } else {
    return "error: unsupported file extension, must be otf or ttf.";
  }
}
18 Replies
Mike
Mike•9mo ago
So then I tried to create it using Bun but got a strange error from GitHub. Running the code locally with Bun it does work which makes it even stranger to me. Here i can use the fontkit package: Code that does not work, running on windmill.dev Bun runtime: 
import * as wmill from "windmill-client@1";
import { Octokit } from "@octokit/rest@20.0.2";
let fontkit = require('fontkit');
import { unlink } from "node:fs/promises";

export async function main(
  fontFile: wmill.Base64,
  fontType: string
) {

  const octokit = new Octokit({
    auth: (await wmill.getVariable('f/COMPANY/GITHUB_TOKEN')),
  });

  const tempFileName = "temp";

  // Write file
  const buffer = Buffer.from(fontFile, 'base64');
  const fontFilePath = `${process.cwd()}/${tempFileName}`;
  await Bun.write(fontFilePath, buffer);

  // Read values
  const font = await fontkit.open(fontFilePath);
  const fontName = await font.postscriptName.toString();
  // This always seems to read as ttf, just asking type for now.
  // const fontType = font.type.toLowerCase();

  // Delete file
  await unlink(fontFilePath);

  const acceptedFileTypes = ["otf", "ttf"];

  if (acceptedFileTypes.includes(fontType) && fontName && fontName !== "") {
    try {
      const result = await octokit.repos.createOrUpdateFileContents({
        owner: "OWNER",
        repo: "REPO",
        path: `PATH`,
        message: `MESSAGE`,
        content: fontFile,
      });
    } catch (error: any) {
      console.log(error.status); // This is a 500 error.
      // The error message is in html and VERY long. Does not reveal any information, only says something is wrong.
      // console.log(error.message);
    }

    const res = await fetch(
      "ENDPOINT TO NOTIFY FILE IS UPLOAD, NOT RELEVANT FOR THIS ISSUE",
    );

    const responseText = res.text();

    return responseText || 500;
  } else {
    return "error: unsupported file extension, must be otf or ttf.";
  }
}
import * as wmill from "windmill-client@1";
import { Octokit } from "@octokit/rest@20.0.2";
let fontkit = require('fontkit');
import { unlink } from "node:fs/promises";

export async function main(
  fontFile: wmill.Base64,
  fontType: string
) {

  const octokit = new Octokit({
    auth: (await wmill.getVariable('f/COMPANY/GITHUB_TOKEN')),
  });

  const tempFileName = "temp";

  // Write file
  const buffer = Buffer.from(fontFile, 'base64');
  const fontFilePath = `${process.cwd()}/${tempFileName}`;
  await Bun.write(fontFilePath, buffer);

  // Read values
  const font = await fontkit.open(fontFilePath);
  const fontName = await font.postscriptName.toString();
  // This always seems to read as ttf, just asking type for now.
  // const fontType = font.type.toLowerCase();

  // Delete file
  await unlink(fontFilePath);

  const acceptedFileTypes = ["otf", "ttf"];

  if (acceptedFileTypes.includes(fontType) && fontName && fontName !== "") {
    try {
      const result = await octokit.repos.createOrUpdateFileContents({
        owner: "OWNER",
        repo: "REPO",
        path: `PATH`,
        message: `MESSAGE`,
        content: fontFile,
      });
    } catch (error: any) {
      console.log(error.status); // This is a 500 error.
      // The error message is in html and VERY long. Does not reveal any information, only says something is wrong.
      // console.log(error.message);
    }

    const res = await fetch(
      "ENDPOINT TO NOTIFY FILE IS UPLOAD, NOT RELEVANT FOR THIS ISSUE",
    );

    const responseText = res.text();

    return responseText || 500;
  } else {
    return "error: unsupported file extension, must be otf or ttf.";
  }
}
and then the simplified code, running locally, that does work: 
import { Octokit } from "@octokit/rest";

let fontkit = require('fontkit');
import { unlink } from "node:fs/promises";
import * as fs from "fs";


const octokit = new Octokit({
  auth: "TOKEN-GOES-HERE",
});

const tempFileName = "temp";

let fontFile: any = fs.readFileSync("ALSHauss-Black.otf");
fontFile = fontFile.toString('base64');

// Write file
const buffer = Buffer.from(fontFile, 'base64');
const fontFilePath = `${process.cwd()}/${tempFileName}`;
await Bun.write(fontFilePath, buffer);

// Read values
const font = await fontkit.open(fontFilePath);
// const fontName = font.postscriptName.toString();
// This always seems to read as ttf, just asking type for now.
// const fontType = font.type.toLowerCase();

// Delete file
await unlink(fontFilePath);

const fontName = "aaa-test";
const fontType = "otf";

const acceptedFileTypes = ["otf", "ttf"];

if (acceptedFileTypes.includes(fontType) && fontName && fontName !== "") {
  try {
    const result = await octokit.repos.createOrUpdateFileContents({
      owner: "OWNER",
      repo: "REPO",
      path: `PATH`,
      message: `MESSAGE`,
      content: fontFile,
    });
  } catch (error: any) {
    console.log(error.status);
    // The error message is in html and VERY long
    // console.log(error.message);
  }

  const res = await fetch(
    "ENDPOINT TO NOTIFY FILE IS UPLOAD, NOT RELEVANT FOR THIS ISSUE",
  );

  const responseText = await res.text();

  console.log(responseText || 500);
} else {
  console.log("error: unsupported file extension, must be otf or ttf.");
}
import { Octokit } from "@octokit/rest";

let fontkit = require('fontkit');
import { unlink } from "node:fs/promises";
import * as fs from "fs";


const octokit = new Octokit({
  auth: "TOKEN-GOES-HERE",
});

const tempFileName = "temp";

let fontFile: any = fs.readFileSync("ALSHauss-Black.otf");
fontFile = fontFile.toString('base64');

// Write file
const buffer = Buffer.from(fontFile, 'base64');
const fontFilePath = `${process.cwd()}/${tempFileName}`;
await Bun.write(fontFilePath, buffer);

// Read values
const font = await fontkit.open(fontFilePath);
// const fontName = font.postscriptName.toString();
// This always seems to read as ttf, just asking type for now.
// const fontType = font.type.toLowerCase();

// Delete file
await unlink(fontFilePath);

const fontName = "aaa-test";
const fontType = "otf";

const acceptedFileTypes = ["otf", "ttf"];

if (acceptedFileTypes.includes(fontType) && fontName && fontName !== "") {
  try {
    const result = await octokit.repos.createOrUpdateFileContents({
      owner: "OWNER",
      repo: "REPO",
      path: `PATH`,
      message: `MESSAGE`,
      content: fontFile,
    });
  } catch (error: any) {
    console.log(error.status);
    // The error message is in html and VERY long
    // console.log(error.message);
  }

  const res = await fetch(
    "ENDPOINT TO NOTIFY FILE IS UPLOAD, NOT RELEVANT FOR THIS ISSUE",
  );

  const responseText = await res.text();

  console.log(responseText || 500);
} else {
  console.log("error: unsupported file extension, must be otf or ttf.");
}
Things i've tried: hardcoding the github token. that made no difference. not using the uploaded file but just uploading a string of text. Also same issue. And uploading a string of text locally does work. What can I do to try and diagnose this issue? really stumped by this... and the error I get is an html error from GitHub, not showing any information. Just saying there is an issue and it has been reported. Makes me thing that something is messed up with the import of Octokit package perhaps?
rubenf
rubenf•9mo ago
What is the error that you see?
Mike
Mike•9mo ago
Github just throws me a 500 error with a generic error message, sadly not much of use: 
<p><strong>Looks like something went wrong!</strong></p>
      <p>We track these errors automatically, but if the problem persists feel free to contact us. In the meantime, try refreshing.</p>
      <div id="suggestions">
        <a href="https://support.github.com/contact">Contact Support</a> &mdash;
        <a href="https://githubstatus.com">GitHub Status</a> &mdash;
        <a href="https://twitter.com/githubstatus">@githubstatus</a>
      </div>
<p><strong>Looks like something went wrong!</strong></p>
      <p>We track these errors automatically, but if the problem persists feel free to contact us. In the meantime, try refreshing.</p>
      <div id="suggestions">
        <a href="https://support.github.com/contact">Contact Support</a> &mdash;
        <a href="https://githubstatus.com">GitHub Status</a> &mdash;
        <a href="https://twitter.com/githubstatus">@githubstatus</a>
      </div>
But the thing that is confusing to me is that in the deno runtime, or locally with Bun, this same request works...
rubenf
rubenf•9mo ago
are you self-hosting?
Mike
Mike•9mo ago
no, using Cloud
rubenf
rubenf•9mo ago
you should try if it works self-hosting then it's just a network difference
Mike
Mike•9mo ago
Maybe a request to GitHub is blocked from your guy's servers or something? Will give self hosted a try and report back Also does not work on self hosted, same 500 error on that request. just running the project through docker with docker-compose
rubenf
rubenf•9mo ago
can you console log the args of the request, local vs windmill
Mike
Mike•9mo ago
Okay i was able to narrow down the issue to one line of code: inside this request: 
      const result = await octokit.repos.createOrUpdateFileContents({
        owner: "OWNER",
        repo: "REPO",
        path: "fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf",
        // path: `fonts/custom-fonts/${fontType}/${fontName}.${fontType}`,
        message: `chore: add font ${fontName}`,
        content: fontFile,
      });
      const result = await octokit.repos.createOrUpdateFileContents({
        owner: "OWNER",
        repo: "REPO",
        path: "fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf",
        // path: `fonts/custom-fonts/${fontType}/${fontName}.${fontType}`,
        message: `chore: add font ${fontName}`,
        content: fontFile,
      });
if i use the hardcoded path the request works, but if i use the string with template literals it doesn't work, am i missing something obvious here? 
// This works
console.log("fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf");
// Output: fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf

// This does not work
console.log(`fonts/custom-fonts/${fontType}/${fontName}.${fontType}`);
// Output: fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf
// This works
console.log("fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf");
// Output: fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf

// This does not work
console.log(`fonts/custom-fonts/${fontType}/${fontName}.${fontType}`);
// Output: fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf
complete script for context: 
import * as wmill from "windmill-client@1";
import { Octokit } from "@octokit/rest@20.0.2";
let fontkit = require('fontkit');
import { unlink } from "node:fs/promises";

export async function main(
  fontFile: wmill.Base64,
  fontType: string
) {

  const octokit = new Octokit({
    auth: "GITHUB-TOKEN-HERE",
  });

  const tempFileName = "temp";

  // Write file
  const buffer = Buffer.from(fontFile, 'base64');
  const fontFilePath = `${process.cwd()}/${tempFileName}`;
  await Bun.write(fontFilePath, buffer);

  // Read values
  const font = await fontkit.open(fontFilePath);
  const fontName = await font.postscriptName.toString();

  console.log(fontName.toLowerCase());

  // This always seems to read as ttf, just asking type for now.
  // const fontType = font.type.toLowerCase();

  // Delete file
  await unlink(fontFilePath);

  const acceptedFileTypes = ["otf", "ttf"];
import * as wmill from "windmill-client@1";
import { Octokit } from "@octokit/rest@20.0.2";
let fontkit = require('fontkit');
import { unlink } from "node:fs/promises";

export async function main(
  fontFile: wmill.Base64,
  fontType: string
) {

  const octokit = new Octokit({
    auth: "GITHUB-TOKEN-HERE",
  });

  const tempFileName = "temp";

  // Write file
  const buffer = Buffer.from(fontFile, 'base64');
  const fontFilePath = `${process.cwd()}/${tempFileName}`;
  await Bun.write(fontFilePath, buffer);

  // Read values
  const font = await fontkit.open(fontFilePath);
  const fontName = await font.postscriptName.toString();

  console.log(fontName.toLowerCase());

  // This always seems to read as ttf, just asking type for now.
  // const fontType = font.type.toLowerCase();

  // Delete file
  await unlink(fontFilePath);

  const acceptedFileTypes = ["otf", "ttf"];
  if (acceptedFileTypes.includes(fontType)) {
    try {

        // This works
        console.log("fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf");
        // Output: fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf

        // This does not work
        console.log(`fonts/custom-fonts/${fontType}/${fontName}.${fontType}`);
        // Output: fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf

      const result = await octokit.repos.createOrUpdateFileContents({
        owner: "OWNER",
        repo: "REPO",
        path: "fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf",
        // path: `fonts/custom-fonts/${fontType}/${fontName}.${fontType}`,
        message: `chore: add font ${fontName}`,
        content: fontFile,
      });

    } catch (error: any) {
      console.log(error.status); // This is a 500 error.
      // The error message is in html and VERY long. Does not reveal any information, only says something is wrong.
      // console.log(error.message);
    }

    // const res = await fetch(
    //   "NOTIFY ENDPOINT",
    // );

    // const responseText = res.text();

    return "test"

    // return responseText || 500;
  } else {
    return "error: unsupported file extension, must be otf or ttf.";
  }
}
  if (acceptedFileTypes.includes(fontType)) {
    try {

        // This works
        console.log("fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf");
        // Output: fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf

        // This does not work
        console.log(`fonts/custom-fonts/${fontType}/${fontName}.${fontType}`);
        // Output: fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf

      const result = await octokit.repos.createOrUpdateFileContents({
        owner: "OWNER",
        repo: "REPO",
        path: "fonts/custom-fonts/otf/BeausiteClassicTrial-Ultralight.otf",
        // path: `fonts/custom-fonts/${fontType}/${fontName}.${fontType}`,
        message: `chore: add font ${fontName}`,
        content: fontFile,
      });

    } catch (error: any) {
      console.log(error.status); // This is a 500 error.
      // The error message is in html and VERY long. Does not reveal any information, only says something is wrong.
      // console.log(error.message);
    }

    // const res = await fetch(
    //   "NOTIFY ENDPOINT",
    // );

    // const responseText = res.text();

    return "test"

    // return responseText || 500;
  } else {
    return "error: unsupported file extension, must be otf or ttf.";
  }
}
rubenf
rubenf•9mo ago
Would recommend using console log to see what differs
Mike
Mike•9mo ago
i did, they are identical. see the message from 4:33 okay im pretty sure template literals inside an object are broken in the Bun runtime on Windmill. This line of code: 
      console.log({test: `fonts/custom-fonts/${fontType}/${fontName}.${fontType}`});
      console.log({test: `fonts/custom-fonts/${fontType}/${fontName}.${fontType}`});
Shows me this output: 
{
  test: "fonts/custom-fonts/otf/\u0000B\u0000e\u0000a\u0000u\u0000s\u0000i\u0000t\u0000e\u0000C\u0000l\u0000a\u0000s\u0000s\u0000i\u0000c\u0000T\u0000r\u0000i\u0000a\u0000l\u0000-\u0000U\u0000l\u0000t\u0000r\u0000a\u0000l\u0000i\u0000g\u0000h\u0000t.otf",
}
{
  test: "fonts/custom-fonts/otf/\u0000B\u0000e\u0000a\u0000u\u0000s\u0000i\u0000t\u0000e\u0000C\u0000l\u0000a\u0000s\u0000s\u0000i\u0000c\u0000T\u0000r\u0000i\u0000a\u0000l\u0000-\u0000U\u0000l\u0000t\u0000r\u0000a\u0000l\u0000i\u0000g\u0000h\u0000t.otf",
}
running it in bun locally, not through Windmill, it does work: this file: 
const fontType = "otf";
const fontName = "aaa-test";

console.log({test: `fonts/custom-fonts/${fontType}/${fontName}.${fontType}`});
const fontType = "otf";
const fontName = "aaa-test";

console.log({test: `fonts/custom-fonts/${fontType}/${fontName}.${fontType}`});
Gives me this output: 
{
  test: "fonts/custom-fonts/otf/aaa-test.otf"
}
{
  test: "fonts/custom-fonts/otf/aaa-test.otf"
}
hmm i think it has to do with the encoding of that string. the fontName does have a typeof of string, but i think it's incorrect utf maybe? The string is read from a otf font file. Just logging the name works but when i place it inside an object it shows all those characters.
rubenf
rubenf•9mo ago
Where is that fontname read from Ok it's read from a file in input Could you send the file here
Mike
Mike•9mo ago
Mike
Mike•9mo ago
okay I now see that this is not an issue with Windmill, I also got this same result running it locally in bun. Something is messed up with the string when using fontkit to read the font name from the file.
rubenf
rubenf•9mo ago
👍 there is very little transformation in windmill on purpose so that issue would have been very weird
Mike
Mike•9mo ago
Sorry about that, will have to see if I can solve this myself! Just one more question, is it possible to get the name and extension of an uploaded file in windmill? im just uploading it like this: 
//
export async function main(
  fontFile: wmill.Base64,
) {
//
//
export async function main(
  fontFile: wmill.Base64,
) {
//
but I believe that just gives you the base64 string, not the filename and extension right?
rubenf
rubenf•9mo ago
yes with the s3 integration but not with base64 see docs about file handling
Mike
Mike•9mo ago
👍