• Mahlzeit@feddit.de
    link
    fedilink
    English
    arrow-up
    5
    ·
    11 months ago

    Took em long enough.

    I wonder if they used ChatGPT to create any of the training data.

    • Eager Eagle@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      11 months ago

      The transformer architecture GPT is based on came from Google. I’m sure the delay has more to do with Google trying to mitigate liability issues that arise with large scale general public usage, and letting OpenAI “test the waters” first.

      • Mahlzeit@feddit.de
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        2
        ·
        11 months ago

        Quite possible. Whatever the case, they apparently saw no pressure to innovate. It implies that tech development is being slowed down by the Big Tech monopolies.

          • Mahlzeit@feddit.de
            link
            fedilink
            English
            arrow-up
            1
            arrow-down
            1
            ·
            11 months ago

            It just seems that Google should have been able to move faster. Yes, they did publish a lot of important stuff, but seeing the splash that came from Stability and OpenAI, they seem to have done so little with it. What their researchers published was important but I can’t help thinking, that a public university would have disseminated such research more openly and widely. Well, I may be wrong. I don’t have inside knowledge.

    • theherk@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      11 months ago

      Not likely. They may have tested it as an adversarial feedback tool, but it would be much more accurate and efficient to get the source data rather than paying OpenAI for maybe correct information.

      They did, I believe, trick ChatGPT into exposing some of its source data though, but it was only a few hundred MB’s.

      • Mahlzeit@feddit.de
        link
        fedilink
        English
        arrow-up
        1
        ·
        edit-2
        11 months ago

        For the fine-tuning stage at the end, where you turn it into a chatbot, you need specific training data (eg OpenOrca). People have used ChatGPT to generate such data. Come to think of it, if you use Mechanical Turk, then you almost certainly include text from ChatGPT.

        • theherk@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          11 months ago

          Yes it could be done that way, and maybe GPT models were used, but calling these API’s isn’t free and there are plenty of open and surely internal models that could be used for that purpose.