Hi Rui.
Sharing and maintaining the integrity of projects is difficult.
From time to time we have to reinstall, things change places, we want to share the project with another person, we want to reorganize our resources differently...
Plugins cannot be included in the project folder.
Correct. But Qtractor warns about missing plugins.
Since we are in a Free Software environment there is no major problem. They can be searched for and installed.
We can even include the installers in the folder to ensure that the session can always be recovered or shared.
But what happens with samples of fluid plugins, samplers... (sf2, sfz, wavs etc)?
Even if we include them, to recover the project we will have to reassign plugin by plugin their samples. Not to mention that we will have to document the project thoroughly and indicate which sample goes in each plugin instance.
Both documentation and recovery can become titanic tasks.
If the samples were shown in the session file it would not be much of a problem. We could edit the file in text mode to search and replace the paths.
But this is not the case, the parameters with the sample file paths are not visible, but compressed.
However, there may be a solution.
Samples with project-relative paths, in case the samples are inside the project folder.
This would make portability much easier. If all the plugins are installed the session should work on any computer, own, shared, past, present, or future.
That is, if the sample is inside the project folder, it does not save the absolute path but the one relative to the project.
If the plugin requires the absolute path, Qtractor has the necessary information to provide it.
I don't know if implementing this is simple, or an odyssey. But it would improve our ability to preserve projects over time and share them.
If implemented, we would only have to take the precaution of working with the sf2, wavs, etc. copied into the project folder. I work with light sf2 files that are between 2 and 100 megabytes. A complete drum kit in ogg for drumkv1 weighs less than a megabyte. I think sacrificing a little space for portability is worth it.
It's just an idea. Greetings.
Just chiming in to point out…
Just chiming in to point out there would be a cost in terms of duplication (multiplied times N). I tend to think "good housekeeping" lends itself better (or at least as good) and will, for the sake of making my point, just describe the portion of my workflow which is geared to address this area of portability.
One bus per instrument named using a short mnemonic string formatted as N. Examples would be NM1, NM2, NM3 (Tal NoiseMaker synth instances), ODIN1, ODIN2, ODIN3 (Odin2 synth instances), SF21, SF22, SF23 (sft2 soundfont instances)... you get the idea. The point is, I know what synths I personally like and tend to work with so I'm always going to know what each of these descriptive labels means.
Track names are used to reference both the bus reference (which we've just established serves as a instrument reference) as well as the name of the patch used. I always place the bus reference alone on line 1 in order to end up with a clean mixer and the patch description goes on line 2.
Basically, those 2 ongoing practices essentially "self document" things in a way that allows me to reverse engineer what's needed should I need to revive a project on a system (maybe a new install) where this or that plugin isn't installed. Of course, that's just going to get me up and running for tracks using a default patch.
If a track ends up using a patch that's been customized, that second line is changed to
Custom
and I take care to go back into the instrument itself and export the patch. The custom patch is exported to a file name like NM1.whatever_extension_is_appropriate and saved to the same directory as the project itself. Now, I've persisted the custom patch and tied it to the project itself...In the case of Soundfonts, we typically store these in a "central" location for good reason (they're potentially quite large). This is where I would not duplicate the thing and store it alongside my project because I should be backing up my soundfonts (and other central artifacts like 3rd party plugin-specific patches, etc) anyway. Again, the ongoing work used in the project layout steps described above serve to "point the way" in the scenario where we need to recreate. Of course, that's just focused on the layout/recovery side of things. When we need to share our things for collaborative purposes, I've broken that discipline and will export (if needed) and/or duplicate a patch or soundfont as needed but that's all part of a more rare scenario as we tend to collaborate less but should always be thinking about recovery.
I also tend to keep my .mid files in their own
session
directory because I don't like a cluttered project directory. I do this by editing the .qtr file itself and modifying the<directory>
node. A quickmkdir session
and moving all .mid files to that location and I'm good to go. Of course, I don't do this right away but soon after a given project feels "legit" and something I know I'll continue to work on. By maintaining a clean project directory, I can maintain aREADME.txt
with any instructions (maybe when I'm collaborating), any custom patches, etc... In other words, those things would be quite difficult to spot if I were to leave dozens or even hundreds of .mid files sitting around. Also, I always create aexport
directory and use it to write files which are exported after a mix-down. Again, keeping a clean root directory yields big wins. Also, I always run the awesomecleanup
function contained in theFiles
right-click menu after a mix-down/export so things are kept nice and clean.Here's just a few quick screenshots showing these easy wins in action.
In general, I'd be cautious about asking for these type of work-flow assumptions to be baked in despite the awesome and well intended inspiration. I'd actually say it's the lack of those work-flow assumptions being baked in that provides me with the ability to do what I need to do in order to address these concerns.
and just to throw a monkey wrench in, it might be super cool if Qtractor used a git repo as a data store?
...I'm running away
re. Improving project portability...
hi,
most of what @G3N-es asks is already in place if and only if:
all other plugin types, store external file paths in their own blob format, so it's almost impossible to track them down anyway--only the plugin itself understands what it's in its own black box--now multiply this for each plugin type you're using... luckyly, some plugins will embed the whole crap of a file contents in the said blob, hugely bloating and impacting the session file size, save and load times... problem solved in these particular cases only, but with a cost.
hth.
cheers
So you did it!
I know you made changes and efforts to achieve this, but I didn't know it had finally been achieved!!!
However, I have tried and have not succeeded.
Although lv2 now allows it, Qtractor seems not to be ready.
I have renamed the qtz file as zip to be able to insert the plugin (in this case gmsynth.lv2).
![](https://www.rncbc.org/drupal/sites/default/files/qtz-conPlugin.png)
To be sure I have inserted into the project folder:
.lv2/gmsynth.lv2, lv2/gmsynth.lv2 and gmsynth.lv2.
It has not worked. Qtractor does not perform the search.
![](https://www.rncbc.org/drupal/sites/default/files/rutas.png)
I have added from a text editor the paths ", " and ", :/" to see if Qtractor would perform the relative search. There has been no success.
This is without taking into account that if we save a qtz, it will only save the resources it knows and is ready to save (midi and wav active in session), and will delete the original qtz generating a new one.
Everything we had stored there will be lost.
P.S
I have had success with uncompressed files.
But I must include the specific path of the file in the Qtractor paths.
Conclusion:
The QTZ approach creates problems and is unnecessary. Better with uncompressed project folders.
Just get qtractor to search for LV2 files by default when we open a project inside the project folder.
Portable plugins issue FINALLY SOLVED :).
However the "samples" issue I mentioned is not solved.
Maybe LV2 contemplates a special protocol for samples, creating a folder with a specific name inside the LV2 folder... I will investigate it.
re. Portable plugins issue...
I'm sorry, but LV2 plugins are NOT identified by their file-system paths whatsoever, but their URIs.
You're just opening a can of worms, so to speak :)
I'm a bit lost, and forgive my lack of knowledge but...
I'm a bit lost, and forgive my lack of knowledge.
Let's go step by step:
1_ LV2 Portables _
What you indicated:
"you save the session to archive/zip file type (*.qtz)."
It doesn't work:
If it works, I don't know the method...
As I indicated above, I've tried everything. (I've also tried with traditional relative paths ./ but that make Qtractor crashes).
Also, as I've indicated, qtz deletes the original qtz when saving, (including all plugins) which makes it a useless methodology.
It does work:
with normal sessions, but you have to add the session path in options for Qtractor to detect it.
So my question is:
Can it be done so that when opening the session, Qtractor always scans the session folder looking for LV2 plugins?
If the answer is yes, the portability of LV2 plugins is solved.
2_ Plugin Samples _
Plugins use absolute paths for their samples.
Qtractor must save these paths in the session file.
But Qtractor can detect if samples are being taken from within the session folder.
It would be something like:
If the samples path contains the session path, save: "./" + ("the samples path" - "the session path").
If a plugin requests samples and the samples path starts with "./" it delivers: "the session path" + ("samples path" - "./").
3_ Making the rest of the plugin formats portable
In fact they already are. Steps:
1 Load the plugin into the session and save the session.
2 Put a copy of the plugin into your project folder.
3 Edit the .qtr in a text editor and change the plugin's absolute path to "./"
4 Delete the original plugin to ensure that we are really dealing with a portable plugin.
Now tested with odin.clap. I already tested it a while ago with VST formats.
It works. And it works without any scanning by Qtractor. It just finds it and loads it.
Then again:
If Qtractor scanned the session folder on opening for plugins, regardless of the format, you could include plugins of any format inside the project folder and it would work.
Because, even if the plugin path inside the session referred to the original absolute path, when it found a copy of the plugin inside the session folder it would use it.
I have tested this as well.
Conclusion:
Qtractor should scan sessions for plugins on opening them.
This makes ALL plugins portable.
As for the samples... you would have to implement what I have indicated.
Wait!!! News
If we tell Qtractor to scan the "." path for CLAP plugins it searches and finds them successfully in the session folder.
We include a "point" as a route. Also works "./" .
I guess it will be the same with the rest... except of course LV2.
If we tell Qtractor to scan the "." or "./" path for LV2 plugins, Qtractor crashes due to a "core violation".
Then:
All formats are portable except LV2
If you explicitly specify the absolute path of the session, it works.
But it is not functional.
Yea, I don't see the upside…
Yea, I don't see the upside to managing plugins in this fashion. Imagine the following scenario...
Over the course of the past year, you've cobbled together a dozen songs; each leveraging a particular plugin. Using this model, each of those now contain a copy of the plugin.
Now you want to update the plugin in order to benefit from a bug fix or new feature... whatever. You have to now hunt down and do the needful across each of these projects. It just doesn't scale at all; especially when we expand this thought exercise to a dozen or more plugins. You'd be better off automating your "centrally installed" plugins via something like Ansible, a shell script, or whatever.
bundling always seems fun but it ends up just sucking
I disagree
If I update a synth plugin, the project will probably not sound the same anymore. Maybe that synth had filters that were too dirty and they fixed it, but it turns out that using that dirty feature as a creative resource.
If they updated a compressor to improve the response means that the project will not sound the same anymore. Etc.
It's about making the project portable and salvageable in the future...
The cost of space doesn't matter now and never did.
There are projects that have great value and you want to keep.
The cost doesn't matter.
I'm not saying that you have to work like this... it's not the way to work. It's the way to "finish art" a project.
"Final art" includes the project, its dependencies and the final Master.
"Final art" projects are usually not saved in work folders, but in drives dedicated to storage.
In the old days, ZIPs, those huge diskettes. Then CDs, DVDs... until today, when the reduction in cost/gigabyte makes it worthwhile to buy (or rescue old ones that you no longer use) an external hard drive for these purposes.
Once you have a "finished art" project, there is no problem recovering it in the future or sharing it, no matter what computer it is. Because a finished art project is autonomous.
The way you described the…
The way you described the option above, about the software scanning locally for the various things does actually make perfect sense. In other words, if the end user chooses to persist relative to the project, use it; otherwise search higher in the list in order to satisfy. Much like how we do all the things in Unix anyway...
While we're here, I've always secretly wished the .qtr file's
<directory>
element was scoped relative rather than as an absolute path. Reason being is that porting to a newly/freshly installed system would force me to recreate the exact path in order to recover. Not saying that's not possible but relative seems to be the move as it just avoids the issue entirely.re. Improving project portability...
Hold your horses!
I now strongly believe you're going into a goose chase...
Write this down, will you?
plugin binaries are not portable between machines or (file-)system images, period.
LV2 plugin URIs must at all times be unique and identify one binary instance and only; the host LV2 inventory makes sure this is the case on all times; working around that (like adding several search paths where one and many might be found, at any given time, will lead to the proverbial undefined behavior, that being only one instance will ever get called and it is uncertain which will be, probably the wrong one (in this regard, it all deeply depends on the libLILV implementation in charge).
so, I would recommend to follow not on this idea--is out of (our/qtractor) control--and put simply, it is not just an uphill battle, rather a predictable lost one.
hth.
cheers
One solution among many
I'm just trying to solve a common problem, but thanks to your explanations I realize that the diagnosis was always wrong.
The problem is not in the plugins.
Qtractor asks you where you host them and everything works.
Backing up plugins, and installables (if necessary) is logical.
This does not prevent portability or conservation over time.
The problem is in those plugins that are dependent on samples, mostly samplers.
And the solution is obvious: Don't use them.
If you have no choice but to use it because you want a very specific sound texture, this is going to be my work method from now on.
If you need a sampler, it's better to use an external tool than a plugin.
Launch the midi sequence from your project to the sampler.
Record the result as if it were a physical instrument.
Include a muted track with the samples used. You can add annotations to the name of each clip if needed.
If you open the project in the future, everything will sound as it should.
If you need to edit the midi sequence, you have the samplers to recreate the instrument. It doesn't matter if it's the same sampler you used or another one.
I think this should work.
I know it's a common problem, because there are samplers (for example Sitala) that have functions to search for lost samples. It's not a solution either, because there will be many samples with the same name and they will have completely different sounds.
So I'm sharing this conclusion in case it helps someone in the future.
Thanks to both of you for your help.
P.S.
In the case of library-based samplers (sf2, sfz...) I simply don't use them. As @windowsrefund pointed out, the size and complexity make them unmanageable.
It annoys me, because I have 3 irreplaceable pianos in sf2, and I can't invest in Pianoteq.
But from now on I'll only use gmsynth.lv2. By the way, it has recently been updated to "GeneralUser GS v2.0.1 by schristiancollins", and as I pointed out if you update some instruments they won't sound the same.
So... that mantra of updating everything isn't always convenient.
Not being able to manage different versions of a program as different programs was and will be a problem.
GN3-es, I don't think you…
GN3-es,
I don't think you need to avoid sf2/sfz at all because of the fact they're so standard. In other words, it's so easy to find a freely licensed thing to play them (Calf Fluidsynth comes to mind but there are obviously several others). As for the actual sounds, this is what I was talking about earlier when I was saying you'd just organize all of these somewhere on your box and simply include that path in your backups. I know backing up this type of data (sound libraries of any kind) can be costly in terms of size but that cost will only get paid the first time. Subsequent backups are going to be incremental assuming you're using something like an rsync or whatever.
The trick is to ALWAYS know exactly where a given plugin is drawing its "patch" from. If you look closely at ODIN2's export functionality, you'll notice it exports your modified patch to its own preferred location. I always drop into a shell and cp or mv that exported patch to the root of my particular project so I have what's needed to rebuild/recreate in the worst case scenario. That level of discipline and workflow best practice yields benefits time and time again across several other plugins I won't bother to list.
All in all, this has been an interesting thread and I do think some good ideas have been brought up. I still wonder if Qtractor's
<directory>
element can be changed to use a relative path rather than absolute. Granted, backwards compatibility can't be sacrificed... I totally get that.Avoiding the problem
I prefer the more radical route. Avoiding the problem.
I have a training project in mind. Teaching music production.
At first it will be just something informal for family and friends.
However, despite the informality, students are still students.
Projects must be shared with students, and students must collaborate with each other.
Avoiding the problem.
Well, sure.. for something…
Well, sure.. for something so generic, I totally get that and appreciate the thought you're putting into the creation of the work. I had assumed you were talking about a more personal work/creation where you'd be making a needless sacrifice.
Add new comment