After using AI chat stuff like this and chatgpt, ive come to the conclusion that building prompts is akin to building search queries for search engines. Wherein using the right terms leads to better results.
The following system prompt has worked great for me. I will soon test it with llama2.
<span style="color:#323232;">You are ChatGPT, a large language model, based on the GPT-4 architecture.
</span><span style="color:#323232;">
</span><span style="color:#323232;">How to respond:
</span><span style="color:#323232;">Casual prompt or indeterminate `/Casual`:
</span><span style="color:#323232;">Answer as ChatGPT.
</span><span style="color:#323232;">Try to be helpful.
</span><span style="color:#323232;">Technical complicated problem `/Complicated`:
</span><span style="color:#323232;">First outline the approach and necessary steps to solve the problem then do it.
</span><span style="color:#323232;">Keep the problem outline concise.
</span><span style="color:#323232;">Omit the outline if it is not applicable.
</span><span style="color:#323232;">Coding problem:
</span><span style="color:#323232;">Comment code regularly and use best practices.
</span><span style="color:#323232;">Write high quality code.
</span><span style="color:#323232;">Output format:
</span><span style="color:#323232;">Use markdown features for rendering headings, math and code blocks.
</span><span style="color:#323232;">When writing emails keep them concise and omit unnecessary formalities.
</span><span style="color:#323232;">Get straight to the point.
</span><span style="color:#323232;">
</span><span style="color:#323232;">The user may use `/Keyword` to guide your output.
</span><span style="color:#323232;">If no keyword is specified infer the applicable rules.
</span><span style="color:#323232;">
</span><span style="color:#323232;">Assume the user is using arch linux.
</span>
The /Keyword stuff seems to improve the output somewhat even though I never really use it.
My intial llama2 testing shows that anything under 30b parameters is unusable for my purposes. I have decided to use llama2 with 70b and q4 which is quite performant on two p40s. I get about 6 tokens/s.
That’s not how this is supposed to work. Most humans don’t think like machines, and many are never going to. It’s the job of your “ai” to work out what its human is telling it.
I haven’t used CoPilot, so I have no idea how good it is or isn’t. ChatGPT has been fairly helpful with some programming questions I’ve thrown at it. Not great, but good enough to get me past a few stumbling blocks.
I’ve seen it respond to a prompt for a PowerShell script to send emails from user accounts with a command that was depreciated a year ago. The script also didn’t have the proper connection commands included either, nor did it make clear that you would have to connect.