Working with KiCAD's terrible library management

KiCAD is a great tool. For the cost of free, you have no limits on board size, layers, pin count, or parts. Sure it does not include an auto router, but who really uses one. In fact, the current build of CERN's additions adds such great automated features to the manual router that you really dont even need an auto router, even on complex boards.

For all the excellent features of the program, there are equally as many frustrating elements. IT is a TEDIOUS program to work with, including as many or more quirks than Eagle. The learning curve is a challange. I recommend this excellent video tutorial series from Contextual Electronics, which covers all the basics, from schematic part creation, schematic entry, footprint creation to PCB layout and finally GERBER output.



But the real agony of using KiCAD for serious work has to do with library/component/module management. It should be noted that file terminology is somewhat clear and may have changed. For reference, a 'Component' in KiCAD terminology is for schematics. A 'Module' is the physical footprint on the PCB, and both are stored in libraries. What seems to be unclear is how and where to store them.

I must preface this by stating that I use Kicad on windows 7. I know that some features work, or work better under linux. I am using an August 2014 build. As of this writing, the "official" windows release is quite old and does not include some of the library features I will mention. However, while the old version is more stable, and slightly more consistant, the problems remaind, and have not improved.

Unlike Eagle, components and modules are not paired together in one file. On one hand, this is great, as you can get to drawing your schematic without consideration for the footprint. You can select and pair up the component from the schematic to any foot print you like. It becomes very easy to select different footprints later, make alterations to the current selection, and save them in the PROJECT, rather than permenantly altering the originals.

So, why the negative press? Because much of the library management is done manually. For instance, let's say you create a new part. You will need to create both the schematic image and the footprint. After doing your schematic drawing, you need to save it either into a new library, or an existing library. But to USE the part in your project, you first need to ENABLE the library. The process is different in Eeschema than in PCBNew.. a fact which makes things all the more difficult; Common tasks among all of the tools are not the same in each tool.

In Eeschema, you enable a library with Preferences/Set Active Libraries. On the dialog, you will need to click on ADD, then browse to the library. It should also be noted that schematic component libraries are usually found in the "/kicad/share/library" folder. You need to do this process for EVERY library you wish to enable for your projects. ONE BY ONE. There seems to be no way to automate the scanning of a folder and adding all libraries that exist there. In fact, adding multiple search paths (bottom half of the dialog) seems to be unhelpful.

Once a library is enabled, it now becomes searchable and parts are now viewable in the normal practice of creating schematics. But the lack of automation is a tedious frustration. Imagine downloading one of the many collection packs with 50 or more libraries. Now imagine spending most of an evening adding each one by hand.



But let's move on to the current footprint (modules in kicad speak) library managment. You will find you have to go through a similar process with PCB footprints. However, the current version broke with the unified dialog that the old version shared with the schematic editor. You still can find it under "Preferences" but rather than being named "Set Active Libraries" It is called "Library Tables"  Worse still is the fact that it is a TOTALLY UNUSABLE piece of garbage.

First, click "Append Row" to add a new row to the table. Click inside the row, and give the library you plan to add a nickname. Next, you will have to MANUALLY TYPE or Copy/Paste the file path. You heard me right. Open up a file browser, find your new library, copy and past the path from the browser to the field of the table, edit the field to include the actual file name. Be sure you get that extension right. It should be .mod for all your old legacy KiCad libraries. Then select the legacy plug-in type.

Now do it again. 100 times. for all you other libraries. Oh my god really?!?!

Ok ok.. so you might have noticed down below that there are some path substitutions to make this process slightly less painful. You STILL need to browse to the file so that you can copy the filename exactly, but you can shorthand the path with something predefined. KISYSMOD for example points to KiCad's system module folder. Or at least it should. You might have to edit that. Go ahead and click in there OHNOYOUCANT!  That:s right. You CANT edit the path shortcuts.

Now, Kicad pros will tell you "everything in Kicad is text files. Just edit the text file directly!" This is an equally short sighted solution. For one, many of the important configureation files are not where you would expect when working on windows. The thing is, you SHOULDNT HAVE TO!

But WTF ever... lets do it.

It took me most of a day seaching the interwebs with no solutions to any of these problems. The few that exist were written by people that assumed you were a power user. Like, "edit this file. Dont know the path to it? Dont care. Im not telling. Find it yourself." So here now, are my hints and tips for editing and setting up recent versions of KiCad!

Getting a recent install of KiCad on a Windows system

One thing you might have noticed about the standard Kicad install, is that it is a very old stable release compared to what most people in linux are working with. Another thing you might notice is that, while it includes all those useful Python scripts, you have no Python interpreter and thus can not use them. I have found absolutely zero helpful info on the python thing. Maybe it is my own RTFM moment, but when seaching for HOW to do something Pi related, every bit of advice skips over all the important bits.. like how to get to a python command line in the first place.

Luckilly, someone solved all of the above and made the manual build process simple. Download the Kicad winbuilder. Follow its instructions carefully, and it will download the latest source code, build it, enable all the latest features, including python and other stuff, and leave you with a working installer. What's more, to update, simply run the builder again. I say to follow the instructions carefully, but honestly it is a one click process, unless you want granular control over what is downloaded, built or installed. One major point to remember is to make sure winbuilder is on a root directory (for ex: "C:KiCad-winbuilder\"). The only downside is that building from source takes a lot of time, and the install folder is significantly larger. My Kicad2013 folder, installed from the standard installer is 418MB. The Kicad winbuilder folder is 4GB.

Set up some folders

Go to your My Documents folder and create a new KiCad folder. If you have an old one, do something like KiCad2014. Dont use spaces. Kicad hates spaces. When migrating old projects and libraries, some things might get mussed up so for the moment, lets not touch your original folder.

Inside this folder make folders for "projects", "symbols-legacy" and "footprints-legacy". Kicad 2013 and older had a habit of using "library" and "modules". Note that library is singular and modules plural. ENGRISH!. Also, it was somewhat ambiguous where files were supposed to go, at least for a user. The new version is still confusing, so lets just start fresh with something that makes sense.

So, copy your old libraries into the new directories. I cant be more clear than that. Mostly because I honestly dont know for absolute certainty. My system ended up being a mess. Good luck to you. You may want to go ahead and make "symbols-new" and "footprints-new" folders as well.

Edit "RunKiCad.bat"

If you do not already have Notepad++, you should install it right away. Open "RunKiCad.bat" (in the root folder of winbuilder) in notepad++

Find SET KIGITHUB=https://github.com/KiCad

This is how we add new shorthand paths for PCBNew. You might notice that KISYSMOD= is unset. Open a browser and find "C:\kicad-builder\kicad\share\modules" This is the module folder installed with KiCad. Copy the path name into RunkiCad.bat.

We can add more shorthands, so go ahead and write one up for your personal modules folder like:

SET MYMOD=C:\Users\Emery\Documents\KiCad\EWP-footprints

That first line refered to GitHub. There is a curated list of modules online which are locked for read only access. By default, these are the only enabled libraries. But the cool thing is that they are automatically updated every time you use KiCad. Just in case you want offline modules, or would like to alter them, the KISYSMOD folder holds copies on your hard drive.

Your own modules should be stored either in the legacy or new folder in My Documents/Kicad. So you should have these paths added to the .bat file.

In addition, Kicad also supports Eagle Ver 6 and newer footprints! So, you can add a shorthand like this:

SET EGMOD=C:\EAGLE-7.0.0\lbr



My set of entries looks like this:

SET KIGITHUB=https://github.com/KiCad

SET KISYSMOD=C:\kicad-builder\kicad\share\modules

SET MYMODLEG=C:\Users\Emery\Documents\KiCad\EWP-footprints-leg

SET MYMODNEW=C:\Users\Emery\Documents\KiCad\EWP-footprints-leg

SET EGMOD=C:\EAGLE-7.0.0\lbr

SET MYEGMOD=C:\Users\Emery\Documents\eagle\lib

Save the file, go to the kicad winbuilder folder and launch kicad by with RunKicad. Get yourself to PCBNew and the Library Table. Right away you notice absolutely nothing ;)

This fix will NOT automatically scan your folders. It simply enables these shorthand path names. In fact, you may not even see new shorthands on the bottom of the dialog until you USE them for the first time. So, go ahead and add a new footprint library as shown above. But dont go crazy yet. I will show you how to bulk add all of the DEFAULT eagle libraries, as well as the offline KICad libraries waiting in your KISYSMOD path in the next section.

Bulk adding standard package libraries

Close out Kicad. Browse to C:\kicad-builder\kicad\share\template

The template folder has several premade files to help setting up KiCad. But it should be noted these are only TEMPLATES. The REAL files are located elsewhere. A fact which confused the dickens out of me for some time.

The table we want to edit in order to add a lot of libraries at once is the "fp-lib-table" Just like Highlander, there can be only one. So start by making a (backup) copy of it ;)

Now, open it in Notepad++. While you are at it, open the other three (fp-lib-table.xxx where xxx is *for Eagle" "for legacy" and "for pretty")

Looking at the files, you start to see the pattern.

What you will need to do is copy and paste all the entries from the the remaining tables INTO the "fp-lib-table". DONT include the final open and closing () bits. Just the entry lines themselves.

BUUUUT!!!!  Ya know.. make sure the files actually EXIST first! Otherwise you have table entries which point to nothing, which could cause errors/crashes/LOOONG delays when loading the table.. ya know.. tedious shit.

Honestly, this whole darn process is tedious.

Save the file.

NOW.... NOW you need to MOVE this longer, improved table to it's proper place. In your file browser, select it and hit control-c.

Next, browse to C:\Users\Emery\AppData\Roaming\kicad

Where "Emery" is your windows user name.

Rename the table there as a backup, and control-v to paste the new file in it's place.

Relaunch Kicad, go to PCBNew, and open the Library Tables. It will take a lot longer to load the first time, but you should have a whole lot of libraries to choose from now :)