-
Notifications
You must be signed in to change notification settings - Fork 16
/
IDEAS
289 lines (233 loc) · 12.5 KB
/
IDEAS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
Guessing unused packages
------------------------
<acme> a heur?stica levaria em considera??o: "idade" do pacote (i.e. h? quanto
tempo ele foi instalado), tamanho, fato de nenhum outro precisar dele, tempo de
acesso a seus componentes, etc
Downloading and committing
--------------------------
- Partial installer may copy to the fixed disk only these entries
for which there are missing pre-requires.
Removable repositories
----------------------
- Ask channel to check if repository is available somewhere.
- Remove packages for unavailable media and ask to replace
it when the current transaction has been committed.
- Removable repositories may be remote as well.
- Use localmedia:// for repositories which may be stored on any
local media (not *one* path like /mnt/cdrom).
- Use a file in the removable media in a standard place
to inform the repository type/position/name/etc.
- Implement autodetection of channels by introducing an
autodetect() function in the channel info file.
- Implement plugins which detect different kinds of
localmedia (autofs, fstab, supermount, etc), with different
notions of mounting/unmounting/etc.
Mirror system
-------------
- Store history of downloads (bytes, download time, errors)
- History queue is shared by all servers, and has a limited lenght
- Compute penality based on stored history (penality >= 0)
- All servers start with penality = 0
- Use servers with the lowest penality
Priority system
---------------
- Packages and channels have priorities
- Both may be overriden by user configuration
- The final package priority is obtained by taking the user
priority for the package, if given, or by taking the highest
channel priority for the given package and summing with the
package priority.
- Upgrades happen always to a package with the same or a higher
priority.
- Downgrades may happen if packages with higher priorities are
found somewhere.
- Priority is used to compute the best choice when multiple options
are available to satisfy a dependency.
Repositories
------------
- Implement an "auto-repository", which downloads the needed info and
uses that info to load one or more repositories (mirrors?).
Remote systems
--------------
- Implement RemoteChannel, RemoteLoader, and RemotePackageManager.
- Implement "daemon" command, which waits for connections, and when
connected, creates a local Control with the given channel
information, loads packages for the given channels, and sends
the loaded packages for being used in a remote loader. It also
handles downloading and committing packages to local a
PackageManager when instructed by a remote Control.
Random
------
- Implement ISO downloading system.
RPM coloring
------------
<jbj> color of a package is union of file colors within.
<jbj> color of a dependency is union of file colors that have the dependency.
<jbj> color of a transaction is union of package colors.
<jbj> %_transaction_color is affinity mask.
<jbj> technically color=1 upgrades color=1, color=2 upgrades color=2. different
arch is incidental.
<jbj> and color for file maps eaxctly to elf32/elf64 magic in file.
<jbj> dependency color is computed, not stored. there are arrays to attch
variable number of dependencies to files, gory but straightforward.
<jbj> so starting pt and count saved for each file, depends dict contains 'P'
or 'R' in 0xff000000 iirc, to determine whether index is into
provides/requires, 0x00ffffff is index into {N,EVR,F} tag arrays.
<jbj> disjoint dependency graphs, each color is it's own graph, not too scary.
<jbj> the far more important issue is that files, not packages, have
dependencies attached. that means a ppkg starts to become a lightweight
container manifest rather than current heavyweight blob.
<jbj> btw, all the arrays are now sorted too, so bsearch becomes possible.
<jbj> hash still better, but rpm always been addicted to bsearch for uglix
portability hysteria ;-)
<jbj> many-to-one is impossible map. one-to-many through filelist, see rpm -q
--file{require,provide}
<jbj> --filerequires displays only DEPENDSDICT(X,N) iff
value & 0xff000000 == 'R'
<jbj> filedependsN is the number of deps starting at filedpendsX indexed
through dependsdict that are attached to a file.
<niemeyer> Hiho!
<jbj> hey!
<niemeyer> Do you have some minutes for a small discussion?
<jbj> sure
<niemeyer> It's about upgrading packages with two different architectures installed in the same machine.
<niemeyer> What do you belive to be the correct way to upgrade these packages?
<jbj> ah, okay ;-)
<niemeyer> Should both architectures be fetcher in parallel?
<niemeyer> fetched
<jbj> there's "correct" and there is what is implemented.
<jbj> the general problem is quite hard.
<niemeyer> I understand that..
<jbj> however, the goal was to smoosh distros for 2 arches together w/o any change.
<niemeyer> What I want is letting users happy with an upgrader that deals with the way rpm handles the problem.
<jbj> that simplifies, so arch1 (or actually color1) upgrades only color1, ditto color2
<niemeyer> So, it's more about what to do, than about how rpm implements it.
<jbj> what to do ... ?
<niemeyer> Yes.. I mean:
<jbj> details please.
<niemeyer> Ok, now I have two packages: .i386.rpm and .ia64.rpm, should I upgrade both at the same time? What if only a new version of the ia64 is available? Should i386 be uninstalled?
<niemeyer> How did you see that (hard) problem when you envisioned support in rpm?
<jbj> ah, so you worry about how to choose what gets upgraded?
<niemeyer> Yes!
<jbj> well traditionally there was archScore() to prefer i586 over i486 on i686.
<niemeyer> Yes.. I'm using that currently.
<jbj> that works for each of two distro arches, the 2 (say i386 and x86_64) are disjoint on elf32/elf64 color.
<jbj> so what is needed is test on color before doing traditional archScore() to choose what gets upgraded.
<jbj> it's basically 2 choices, one choice for each color.
<niemeyer> Is there an easy way to do that?
<niemeyer> (test on color)
<jbj> and then throw the choices into same or different transactions.
<jbj> sure, there are methods to get the color of all rpm objects.
<jbj> the color of an object is just the union of the file colors contained/attached in the object.
<niemeyer> So I can just ask, using the rpm API, for the color of a header, for instance?
<jbj> so a pkg that has an elf32 file is color = 1
<jbj> headers are not rpm objects ;-)
<niemeyer> Are they not? :)
<jbj> but headerGetEntry on file colors, or the ints together. that is the header/package color.
<jbj> headers are being carefully yanked out of the rpmlib API, rpmlib uses ds/fi/te etc everywhere possible.
<jbj> yes, rpmtsAddInstallElement and rpmdbMatchIterator still use header, it ain't exactly easy abandoning the most important data structure in rpm.
<niemeyer> So there's no way to ask the color of a package but going through the maps of colors in the package and compute the per-file color?
<jbj> nope. files have color, all other colors are computed, to localize in case I need to change something.
<niemeyer> Btw, I still have our last conversation on colors here. I promise I won't bother you with the same questions. :)
<jbj> np, you'll be back for more when you discover ia64 hackery ;-)
<niemeyer> Hehehe :)
<niemeyer> Well.. I guess I'll have to figure out how to compute the color of a package using file colors then.
<jbj> headerGetEntry on FILECOLORS, loop over array color |= fcolor[i]
<jbj> color &= 0x3
<jbj> that is pkg color
<jbj> hdr color too
<niemeyer> Cool.. will use that. Thanks!
<niemeyer> Then, I'll have to attach a color to every package, and figure upgrades using a per-color logic, as you suggested.
<niemeyer> Ok.. I think I'm starting to see the light. :)
<niemeyer> Now, let's get into per-color dependencies..
* niemeyer runs..
Package list sorting
--------------------
<jbj> niemeyer: just the voices in my head. ther is 1 degree of freedom for
tsort. FIFO/LIFO are perfectly ok, but the goal of ordering in an install is to
create as many leaf nodes as soon as possible, number of immediate descendants
in sub-tree is/was easy weight (because available in Knuth's tsort algorithm).
better weight is number of nodes in sub-tree, not just descendants. --chainsaw
seemed obvious thing to do
<jbj> heh, stare at mutisets, think about permutation groups and cycles.
multiply inherited dependencies (i.e. node has multiple possible "parents") is
then a forbidden permutation on the permutation group.
Debian repository
-----------------
Fix of the "whois"-package to work with the new registrar of the .org-domains
(Added 2003-01-31, last checked 2004-04-16) (Download as text) - maintained by
dreamind at dreamind dot de
Packages: whois
Architectures: i386
deb http://www.dreamind.de/debian/ woody-fixes main
deb-src http://www.dreamind.de/debian/ woody-fixes main
Files:
http://www.br.debian.org/mirror/list
http://download.unesp.br/linux/debian/
http://www.dreamind.de/debian/dists/woody-fixes/main/binary-i386/Packages.gz
http://www.dreamind.de/debian/dists/woody-fixes/main/binary-i386/Packages
http://www.dreamind.de/debian/dists/woody-fixes/main/binary-i386/whois_4.5.25-1_i386.deb
Manual of dpkg:
http://sistemac.carnet.hr/paketi/dpkg-full/
About ordering:
http://lists.debian.org/deity/1998/03/msg00036.html
Debian Policy:
http://www.debian.org/doc/debian-policy/
Slackware repository
--------------------
slackpkg: http://slackpkg.sourceforge.net/
swaret: http://www.swaret.org/
slapt-get: http://software.jaos.org/
http://www.linuxtux.org/~juhl/slackpacks/
ftp://ftp.nluug.nl/pub/os/Linux/distr/slackware/slackware-current/
ftp://ftp.slackware.com/pub/slackware/slackware-current/
Mandrake repository
-------------------
ftp://ftp.planetmirror.com/pub/plf/mandrake/cooker
Red Carpet repository
---------------------
http://open-carpet.org/
ftp://ftp.ximian.com/pub/
http://rpms.subpop.net/channels.xml.gz
YUM repository
--------------
http://download.fedora.redhat.com/pub/fedora/linux/core/development/i386/repodata/
Examples of cases where the upgrading algorithm is better
---------------------------------------------------------
- Package A requires a dependency provided by B-1.0. B-1.0
is then upgraded to B-2.0, but the dependency needed by A
is moved to package C. When an upgrade is done, will
package B-1.0 be upgraded to B-2.0 and C be installed,
or will A be removed?
(http://bugzilla.atrpms.net/show_bug.cgi?id=160)
- Package A requires an implicit dependency, which is provided
by package B, C and D. OTOH, package A has an explicit
dependency on package C. Will package B and D be selected
for installation, even though the dependency is satisfied
by C, which must necessarily be installed?
- Package A is obsoleted by B and C, and they don't conflict.
Will package B and C be selected for installation?
High level interface
--------------------
- Implement a higher level interface for package selection.
http://bugzilla.conectiva.com.br/show_bug.cgi?id=11124
http://fedora.redhat.com/projects/anaconda-installer/images/screenshot-0002.png
http://people.debian.org/~mvo/synaptic/ideas/synaptic-app-idea2.png
Parallel support
----------------
http://www.happyassassin.net/2005/05/04/a-quick-guide-to-urpmi-parallel/
Alternative logic for priorities?
---------------------------------
<Insount> You know, I think the most common use scenario is:
<Insount> 1. By default, track the latest version from channel 'stable'.
<Insount> 2. If don't find the package there, look for it in in 'unstable' or 'extra' or 'superextra', in this order. And track the latest version you find in the topmost amond these.
<niemeyer> Insount: What if a relation can't be solved without a package from an auxilar channel?
<Insount> 3. For packages foo and bar, track the version in 'unstable' (even if they're in 'stable'), and get the minimal dependencies from there too.
<Insount> EOF
<Insount> niemeyer: Bring in whatever packages you need, but always pick the first feasible combination, judging only by priority at the forking point rather than the whole combination.
<Insount> niemeyer: That is, search the tree of combinations in depth-first order, ordering by priority and each node, and return the first feasible combination.
<Insount> s/and/at/
<niemeyer> Insount: What about the choices that follow from the possibility of bringin in a package from the alternative repository?
<Insount> niemeyer: Bring them in, choosing the "best" version for each -- judging by the priorities for that package alone.
<Insount> niemeyer: For each package, you try versions in decreasing priority order. For each of them, add all dependneices (recursion). Once you have a consistent set, stop.
vim:ts=4:sw=4:et