IML 6.3.0

Igor Pashev (5):
Reconfigure zfs-zed before start zedenhancer (#2341)
Implement GUI for Stratagem reports (#2334)
Allow different CLI names for Agent CLI (#2368)
Require influx < 2 (#2377)
Use cursor pointer on the Management menu (#2423)

Joe Grund (49):
Add devpath to targets (#2319)
Convert stratagem scanning over to graphql (#2328)
Fix Vbox fencing (#2342)
Update iml-action-client (#2340)
Fixup changed structure path (#2345)
Stop client changes when adding / removing servers (#2348)
Extract a few common postgres fns (#2354)
Update action-client (#2356)
Fix Bug in generic file scan (#2357)
Remove chroma cli (#2365)
Update crypto to support newer openssl (#2371)
move some python CI checks to github actions (#2369)
Update graphql python client (#2372)
Add list target CLI
Add LISTEN / NOTIFY for target table
Duplicate target insert during attempted target mount (#2381)
Extract device-scanner client (#2378)
Update resource ban query to filter nulls (#2394)
Do not try to parse empty lnetctl result (#2400)
Fix lnetctl not found (#2402)
Use direct lctl device calls for snapshots (#2364)
Update client.rs (#2407)
Use target table in GUI
Trim python target API
Use graphql to get active hosts
Remove extra checks from stratagem config
Remove ManagedTargetMount parts
Refactor fs detection
Add precommit with rusty-hook (#2409)
Add iml-ssh crate (#2410)
Commit clippy --fix changes (#2411)
Switch to using the journald driver for docker logging (#2412)
Remove configure-docker-network-trigger (#2413)
Add some extra tracing to the wbem-client (#2415)
Always use adm node as ntp server (#2414)
Filter identical record changes from warp-drive (#2418)
Skip identical daemon plugin updates (#2419)
Include MGS fs info with devices (#2420)
Bump sqlx to latest (#2422)
Silence python http agent logging (#2424)
Use kill_on_drop for all agent commands (#2426)
Fix more clippy issues (#2429)
Ensure iml-docker starts after multi-user.target (#2433)
Switch all systemctl preset to systemd_post (#2431)
Support combined MGS / MDT (#2430)
Action plugin execution blocks daemon plugins (#2436)
Update docker setup (#2444)
Fixup copy-embedded-settings (#2448)
Fix client mount generation (#2450)

Michael Pankov (7):
Switch UI to new logs GraphQL endpoint (#2336)
Error on unknown FS (#2350)
Implement server profile GraphQL queries (#2363)
Switch server profiles CLI to GraphQL (#2376)
Search for config in different places (#2387)
Add writing out of config (#2388)
Remove CLI docker bits (#2398)

Nathaniel Clark (11):
Fix quoting for graphql query from FS (#2339)
Update lamigo/lpurge for lipe compatibililty (#2292)
Add add/remove_fstab_entry agent actions (#2352)
task-runner: Decrease verbosity of debug level (#2346)
Return all resources from iml-agent::high-availability::get_crm_mon (#2347)
agent: Return cloned resources with index from crm mon output (#2370)
Add create and destroy resource (#2178)
Update iml-system-*-tests/Cargo.lock (#2390)
Add ha_resource_move action (#2391)
agent: Fix clippy warnings (#2405)
Add Basic formatting pre-commit check (#2417)

Nick Linker (1):
Implement the fix (#2134)

Will Johnson (10):
Begin converting the linux plugin over to rust (#2321)
Integrate network interface data into IML (#2353)
Fix Stale Influx Entries (#2373)
Add fs_type field to targets table (#2343)
Ingest lnet data from storage servers into management db (#2362)
Add a graphql endpoint to construct the clientmount command. (#2392)
Fix unit tests (#2395)
Add rust rpm provisioner to adm node (#2382)
Handle Resource Bans on Filesystem Detail Page (#2408)
Bump to 6.3 (#2434)

beevans (4):
EX-1979 filesync restore (#2367)
EX-2304 filesync: create the right path (#2449)
Fix for issue 2446 (#2447)
EX-2312 cloudsync: Fix cloudsync lipe config (#2451)

IML 6.2.0

IML 6.2.0

Changes

Igor Pashev (26):
Fix file descriptor leak in the postoffice plugin (#1989)
Monitor client mounts in iml-device (#1911)
Handle sessions in one place (#2026)
Improve and unify server operations in CLI (#2040)
Fix "server add" invocation in tests (#2053)
Fix OST pool grow (#2064)
Agent to teardown sessions on signals (#2077)
Fix client mount in run_stratagem (#2085)
Show command modal when modifying file scan (#2095)
Deduplicate lctl calls in iml-agent (#2099)
Rename try_mount to mount in iml-agent (#2104)
Make listing servers default command (#2042)
Implement creating and destroying snapshots (#2106)
Implement listing snapshots (#2145)
Implement snapshot listing daemons (#2205)
Do not warn if snapshot listing cannot be found (#2224)
Simplify Snapshot structure (#2222)
Implement snapshot listing in CLI (#2234)
Add snapshot interval operations to CLI (#2257)
Add snapshot retention ops to CLI (#2267)
Implement GUI for snapshot retention policies (#2279)
Use three dashes instead of two (#2280)
Update calculation of free/used space in IML CLI (#1803)
Add shell completions for CLI (#2285)
Allow different names for IML CLI (#2305)
Add CLI for stratagem reports (#2323)

Joe Grund (118):
Fixup IML 6.1 repo path (#1970)
Add missing sfa table (#1973)
Fixup controller monitoring (#1974)
Allow for empty EnumerateInstances (#1980)
Update influxdb healthcheck (#1986)
Add wait-for-settings check (#1987)
Update rabbit healthcheck (#1990)
Fix branch of _test_yum_rpm_sanity that returns a tuple of bool (#1995)
Add iml-agent README (#1993)
remove SRCMAP_PATH (#1996)
Add resource bounds for influx and rabbitmq (#1992)
Create a command to update docker services (#1997)
Remove blocking debug in postoffice (#2002)
Add in_flight handle to active sessions (#2001)
Update lapin to latest (#1972)
Do not set environment variables in tests (#2008)
IML cannot detect filesystem when a client is mounted on a storage server (#2018)
Fix user disconnects (#2006)
Use systemd_post instead of systemctl preset (#2022)
Move device-scanner into the IML repo (#2009)
Use sqlx for iml-devices (#2025)
Update retry_future to take ownership of policy (#2029)
Fix fs detail overflow (#2038)
Do not install postgres-devel (#2049)
Opt-out of NetworkManager for management network (#2055)
Only unmount clients that were on the current host (#2045)
bump deps to latest (#2058)
Move Stratagem config to the top (#2056)
Convert syslog service into Rust (#2062)
Bump deps (#2076)
Remove queue purge during startup (#2080)
Specify docker version (#2082)
Update mem::replace assignments (#2081)
Move snapshot location in agent testing (#2083)
Fix Journal cursor deadlock (#2092)
Use docker buildx for building (#2101)
Make docker image tarball a required file (#2103)
Only destroy matching sessions (#2105)
Create codeql-analysis.yml (#2112)
Rename client nodes to non hexadecimal characters (#2113)
Add all packages to all target (#2120)
Bump sqlx to latest (#2116)
Use a single lustre install step (#2124)
Extract remote_action code from sender (#2123)
Remove ObjectCache from ClientMount (#2133)
Remove parsing of jobstats (#2136)
Fix firefox tests (#2138)
Allow local actions to make db calls (#2141)
Move client mount / unmount to rust (#2142)
Allow for management of existing resources (#1919)
Fixup client mounts (#2148)
Task mailbox (#2150)
Remove iml-orm changes from task queue
Create iml-report service
Ensure upstream repo is used for docker (#2151)
Ensure active_workers are released (#2155)
Add a bunch of debug to the task-runner. (#2164)
Switch iml-orm to sqlx in iml-cli (#2162)
Don't use try_acquire alongside spawned task (#2165)
Fix log deletion (#2168)
Fix missing space in fstab client mount (#2169)
Ensure empty snapshots do not produce parse error (#2170)
Introduce iml-change crate (#2171)
Build up targets from device-scanner output (#2107)
Set sqlx::query to warn (#2174)
Update sfa integration to use sqlx (#2172)
Fix docker-compose.yml (#2183)
Implement a crm_mon XML reader (#2182)
Fix migration help text (#2185)
Add corosync daemon plugin to Rust (#2184)
Implement Experimental GraphQL support (#2186)
Add a graphql endpoint to list targets (#2188)
Support Rust 1.46 (#2193)
Fix error handling syntax (#2190)
Remove Device scanner make command from local build (#2192)
Use non-github deps for libzfs crates (#2195)
Add test_setup fn to iml_postgres (#2206)
Use IML cli inside iml-timer (#2202)
Bump black to latest (#2207)
Bump nginx to support tlsv1.3 (#2217)
Pin vine to 1.3.0 (#2223)
Add corosync realtime monitoring (#2204)
Add deadline support for daemon plugins (#2221)
Add clush to vagrant (#2227)
Add Snapshots to iml-warp-drive (#2225)
Bump libloading to latest (#2228)
Add barrier support to snapshot create (#2235)
Add iml-graphql-queries crate (#2236)
Implement graphql queries for all snapshot queries and mutations (#2237)
Finish snapshot management CLI (#2238)
Add snapshot name validation
Hardcode docker compose version (#2245)
Add snapshot management GUI (#2243)
Add snapshot configuration API (#2247)
Fix docker releases so older compose is used (#2252)
Add list snapshot interval GUI (#2253)
Create add interval modal (#2256)
Fix add_interval.rs (#2264)
Extract command polling logic to it's own crate (#2251)
Implement influx client extension (#2262)
Implement snapshot interval deletion support in GUI (#2266)
Fix Stat deletion (#2263)
Fix parse (#2272)
Fix Limit (#2274)
Fix bad merge
Write timer service to non volume mounted area (#2284)
Remove CLI indirection (#2294)
Use default-tls for wbem-client (#2301)
Fix enable snapshot race (#2303)
Remove "Other" bucket (#2306)
Bump versions for new release. (#2309)
Account for empty ARRAY.VALUE (#2311)
Bump lustre-collector (#2313)
Remove spinners lib from manager cli. (#2315)
Change reserve_value to doc comment (#2324)
Add pbr as yum dep (#2333)
Add graphql schema listing support (#2316)
Remove swap-emitter.timer (#2335)

Michael Pankov (19):
Allow for influx retention time to be tunable (#2046)
Drop retention policy before creating it (#2051)
Use Identity::from_pem (#2054)
Switch direct dependencies to rustls (#2073)
Disable default features of reqwest (#2098)
Implement mounting and unmounting snapshots (#2109)
Move ActionType to iml_wire_types (#2128)
Switch the iml-rabbit to rustls (#2115)
Add FS name constraint (#2047)
Implement Mount Step (#2121)
Implement Create/Destroy SnapshotStep (#2140)
Implement MountSnapshotJob / UnmountSnapshotJob (#2156)
Ignore .env (#2179)
Forward Postgres port (#2180)
Implement Snapshot listing GraphQL API (#2210)
Implement entire Snapshot GraphQL API (#2177)
Implement idle snapshot autounmount (#2246)
Lower snapshot listing interval (#2276)
Use transaction for table update (#2277)

Nathaniel Clark (30):
Drop Continuous Queries for influx before create (#1971)
grafana: Fix usage of "filesystem" where it should be "fs" (#1976)
vagrant: Rename client lustre network target (#2004)
Task Mailbox 0016 migration changes (#2016)
Ensure activation of LVs during creation (#2028)
Fix agent ostpool resend (#2037)
vagrant: Add client mount provisioner (#2078)
Update iml-manager-env for docker and postgres (#2074)
Add alternate client mount provisioners (#2094)
tests: Mount filesystem(s) during detect_fs (#2102)
Add TAGS to gitignore (#2110)
Tests: Add clients to ldiskfs zfs profile list (#2111)
Add "make check" and "make fmt" to Makefile (#2087)
Split FilesystemDetect state into 3 (#2118)
action-client: Add invoke_rust_agent (#2122)
Tests: Unify iml-configured graph node (#2130)
Fix cargo formatting of system tests (#2131)
Add 'iml debugapi' to iml cli (#2132)
Quote arguments passed into docker iml (#2143)
debuapi: Optionally read from stdin for Body (#2149)
Make db pool size configurable (#2158)
Add ops to ResourceAgentType (#2166)
Add Mirror actions for HPv2 (#2153)
agent: Remove duplicate function (#2181)
agent: Add start/stop resource to rust agent (#2146)
Add debugql command to talk to /graphql endpoint (#2209)
Make task remove idempotent (#2260)
graphql: Return fqdn from get_fs_cluster_hosts() (#2287)
chroma: Add graphql helper function (#2290)
Pass RUST_LOG to cli (#2293)

Nick Linker (3):
Purge loop (#2214)
EX-1687 graphql commands (#2240)
Implement fetching commands by ids (#2265)

Will Johnson (35):
Use the filesystem from the object cache when unmounting / mounting (#1978)
Add the configure_ntp provisioner to client nodes (#1977)
Fix policy index in sfa (#1979)
Update footer branding (#1982)
Fix mount jobs such that they use the correct mount point (#1981)
Make chart icons ...

6.2 dev

Make db pool size configurable (#2158)

* Make db pool size configurable

Allow override of POOL_LIMIT on a per service basis
Adjust most limits down
Adjust task-runner limit up to 10

Fixes #2157

Signed-off-by: Nathaniel Clark <[email protected]>

* Fix spelling and add LIMIT to task-runner

Signed-off-by: Nathaniel Clark <[email protected]>

* update sqlx-data.json

Signed-off-by: Joe Grund <[email protected]>

* Address review comments

Signed-off-by: Nathaniel Clark <[email protected]>

Co-authored-by: Joe Grund <[email protected]>

IML 6.1.0

IML 6.1.0

IML 6.0.0

IML 6.0.0

6.next: Lower Active alerts (#1789)

When an alert is lowered it is not reflected in the IML GUI.

This is because we need to call `update_activity_health` in the
`Msg::RemoveRecord` variant.

Signed-off-by: Joe Grund <[email protected]>

IML 5.1.0

We are pleased to announce general availability of IML 5.1.0

IML 5.1 adds support for Lustre 2.12.3 and features a number of enhancements and bugfixes.

General documentation can be found here.

Issues can be reported here.

Changes from 5.0 can be found here

How to get it

IML 5 can be installed as a set of RPMs via Fedora Copr.

To install the software via RPM, follow these steps.

To upgrade from IEEL 2.4.x / 3.1.x, IML 4.0.x or IML 5 to IML 5.1 follow these steps.

• IEEL 2.4
• IEEL 3.1
• IML 4.0.x
• IML 5.0.x

Upgrades from IEEL versions need to upgrade to IML 4 then IML 5.

For instance:

• Upgrading from IEEL 2.4 to IML 5, perform an upgrade from IEEL 2.4 -> IML 4.0.10 -> IML 5.1
• Upgrading from IEEL 3.1 to IML 5, perform an upgrade from IEEL 3.1 -> IML 4.0.10 -> IML 5.1

Features

• Adds ability to fully deploy storage servers in parallel with the IML CLI. Example:

  iml add --hosts mds[1,2],oss[1,2] --profile base_managed_patchless

This will deploy managed servers to each node in the hostlist expression.

Enhancements

• Lustre 2.12.3 support
• Managed Mellanox server support (#955)
• Introduction of Rust plugin framework for messaging between manager and agents.
• Introduction of iml cli on the manager node.
• Introduction of iml-agent cli on the agent nodes.
• Improve CPU utilization on the IML manager node (#1040)
• Refresh the filesystem page using WebAssembly (#1072)
• Switch from using F# for device-scanner to Rust (#236)

Bugfixes

• Ensure chronyd is stopped and ntpd is running in managed mode (#1008)
• Keep IML fs state available if MDT0 is mounted (#1023)
• Fix re-register of existing profile (#1086)
• Only show zpool on volumes page when backed by mpath (#742)
• Wait for incoming device data on manager side (#1036)

Known Issues

These are known issues that are being actively worked. Fixes will be pushed to the IML 5 repo as they land.

• Rabbitmq may not start upon reboot (#480)
  • Workaround If the manager is not running upon reboot, issue the following command to restart rabbitmq and all IML services: systemctl restart iml-manager.target
• ZFS backed nodes may not reboot upon power loss
  • Workaround Boot the node
• New OSTs need to be added via cli (#1101)
  • Workaround Use the chroma cli on the manager node to add new OSTs. Ex:
    chroma --username=<USERNAME> --password=<PASSWORD> ost-add --filesystem=<FS_NAME> <HOST_NAME>:<DEV_PATH>

5.next: Add trailing newline to each 1000 fid chunks. (#1131)

Fixes #1130.

Signed-off-by: Joe Grund <[email protected]>

IML 4.0.10.2

IML 4.0.10.2

IML 4.0.10.2 RC1

IML 4.0.10.2 RC1

Not for production use.