LocalLLaMA

2244 readers

1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

pax@sh.itjust.works

SkySyrup@sh.itjust.works

noneabove1182@sh.itjust.works

Fine tuning and datasets questions (sh.itjust.works)

submitted 1 year ago by Slacking@sh.itjust.works to c/localllama@sh.itjust.works

0 comments fedilink hide all child comments

I'm building an instruct dataset but I'm not sure if I should replace some with general conversation instructions.

What's the consensus right now on the best ratio of data? I saw Lima even had r/writingprompt in there.

Basically, how do you guys structure your data when fine tuning and what do you add other then the basic instruct data. Ratio of code questions seems to be important as well.

The amount in the different datasets is all over the place too, what do you guys aim for?

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here