Inside the GPT-3

  • Behohippy@lemmy.world
    link
    fedilink
    arrow-up
    3
    ·
    1 year ago

    I’ve got a background in deep learning and I still struggle to understand the attention mechanism. I know it’s a key/value store but I’m not sure what it’s doing to the tensor when it passes through different layers.