Initial commit
This commit is contained in:
Zhongwei Li
238
skills/iot-edge-module/references/deployment-manifests.md
Normal file
238
skills/iot-edge-module/references/deployment-manifests.md
Normal file
@@ -0,0 +1,238 @@
|
||||
# IoT Edge Deployment Manifests Reference
|
||||
|
||||
This document provides reference information for the deployment manifest structure used in this project.
|
||||
|
||||
## Manifest Types
|
||||
|
||||
Projects typically use one or more deployment manifests to organize modules:
|
||||
|
||||
### Example: Base Deployment Manifest
|
||||
|
||||
**Naming pattern**: `*.deployment.manifest.json` (e.g., `base.deployment.manifest.json`)
|
||||
|
||||
**Purpose**: Contains modules that should be deployed to all or specific edge devices.
|
||||
|
||||
**Example modules**:
|
||||
- Metric collector modules - Collects operational metrics
|
||||
- Telemetry transformation modules - Enriches raw telemetry
|
||||
- Custom business logic modules
|
||||
|
||||
**Routing**:
|
||||
- Modules typically route to `$upstream` (IoT Hub) or to other modules via BrokeredEndpoint
|
||||
|
||||
## Deployment Manifest Structure
|
||||
|
||||
### Top-Level Structure
|
||||
|
||||
```json
|
||||
{
|
||||
"modulesContent": {
|
||||
"$edgeAgent": { ... },
|
||||
"$edgeHub": { ... }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Module Definition ($edgeAgent)
|
||||
|
||||
Each module is defined under `$edgeAgent` with the following structure:
|
||||
|
||||
```json
|
||||
"properties.desired.modules.<modulename>": {
|
||||
"version": "1.0",
|
||||
"type": "docker",
|
||||
"status": "running",
|
||||
"restartPolicy": "always",
|
||||
"startupOrder": <number>,
|
||||
"settings": {
|
||||
"image": "${MODULES.<modulename>}",
|
||||
"createOptions": {
|
||||
"HostConfig": {
|
||||
"LogConfig": {
|
||||
"Type": "json-file",
|
||||
"Config": {
|
||||
"max-size": "10m",
|
||||
"max-file": "10"
|
||||
}
|
||||
},
|
||||
"Binds": [
|
||||
// Optional host volume binds
|
||||
],
|
||||
"Mounts": [
|
||||
// Optional volume mounts
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"env": {
|
||||
// Optional environment variables
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key fields**:
|
||||
- `version`: Module version (typically "1.0")
|
||||
- `type`: Always "docker"
|
||||
- `status`: Desired status ("running")
|
||||
- `restartPolicy`: Restart behavior ("always")
|
||||
- `startupOrder`: Module startup sequence (lower starts first, system modules use 0)
|
||||
- `image`: Container image (use variable substitution: `${MODULES.<modulename>}`)
|
||||
- `createOptions`: Docker container creation options
|
||||
- `LogConfig`: Log rotation settings (10m max size, 10 files)
|
||||
- `Binds`: Host path bindings for persistent storage
|
||||
- `Mounts`: Named volume mounts
|
||||
- `env`: Environment variables (use variable substitution for secrets)
|
||||
|
||||
### Routing Configuration ($edgeHub)
|
||||
|
||||
Each route is defined under `$edgeHub`:
|
||||
|
||||
```json
|
||||
"properties.desired.routes.<routename>": {
|
||||
"route": "<route expression>",
|
||||
"priority": 0,
|
||||
"timeToLiveSecs": 86400
|
||||
}
|
||||
```
|
||||
|
||||
**Common route patterns**:
|
||||
|
||||
1. **Module to IoT Hub (upstream)**:
|
||||
```json
|
||||
"route": "FROM /messages/modules/<modulename>/outputs/* INTO $upstream"
|
||||
```
|
||||
|
||||
2. **Module to module (BrokeredEndpoint)**:
|
||||
```json
|
||||
"route": "FROM /messages/modules/<sourcemodule>/* INTO BrokeredEndpoint(\"/modules/<targetmodule>/inputs/<inputname>\")"
|
||||
```
|
||||
|
||||
**Key fields**:
|
||||
- `route`: Route expression using IoT Edge routing syntax
|
||||
- `priority`: Route priority (0 = normal)
|
||||
- `timeToLiveSecs`: Message TTL (86400 = 24 hours)
|
||||
|
||||
### Variable Substitution
|
||||
|
||||
Manifests use variable substitution for dynamic values:
|
||||
|
||||
- `${MODULES.<modulename>}` - Module container image URI
|
||||
- `${ContainerRegistryUserName}` - Registry username
|
||||
- `${ContainerRegistryPassword}` - Registry password
|
||||
- `${ContainerRegistryLoginServer}` - Registry server URL
|
||||
- `${LogAnalyticsWorkspaceId}` - Log Analytics workspace ID
|
||||
- `${LogAnalyticsWorkspaceSharedKey}` - Log Analytics shared key
|
||||
- `${IotHubResourceId}` - IoT Hub resource ID
|
||||
|
||||
## Module-Specific Patterns
|
||||
|
||||
### Modules with Volume Mounts
|
||||
|
||||
Use named volumes for module-specific storage:
|
||||
|
||||
```json
|
||||
"Mounts": [
|
||||
{
|
||||
"Type": "volume",
|
||||
"Target": "/app/data/",
|
||||
"Source": "<modulename>"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### Modules with Host Binds
|
||||
|
||||
Use host binds for shared storage or device access:
|
||||
|
||||
```json
|
||||
"Binds": [
|
||||
"/srv/aziotedge/opc/opcpublisher/:/app/opc/opcpublisher/",
|
||||
"/dev/tpm0:/dev/tpm0"
|
||||
]
|
||||
```
|
||||
|
||||
### Modules with Privileged Access
|
||||
|
||||
For modules requiring device access (e.g., TPM):
|
||||
|
||||
```json
|
||||
"Privileged": true
|
||||
```
|
||||
|
||||
### Modules with Environment Variables
|
||||
|
||||
Configure modules via environment variables:
|
||||
|
||||
```json
|
||||
"env": {
|
||||
"OptionsClass__PropertyName": {
|
||||
"value": "value or ${VariableSubstitution}"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Adding a New Module to a Manifest
|
||||
|
||||
To add a new module to a deployment manifest:
|
||||
|
||||
1. **Add module definition to `$edgeAgent`**:
|
||||
- Use `properties.desired.modules.<modulename>` as the key
|
||||
- Set appropriate `startupOrder` (consider dependencies)
|
||||
- Set `image` to `${MODULES.<modulename>}`
|
||||
- Configure `createOptions` (log rotation, binds, mounts)
|
||||
- Add environment variables if needed
|
||||
|
||||
2. **Add routing to `$edgeHub`**:
|
||||
- Use descriptive route name: `properties.desired.routes.<modulename>ToIoTHub`
|
||||
- Set route expression based on message flow
|
||||
- Use standard priority (0) and TTL (86400)
|
||||
|
||||
3. **Update system properties** (only if needed):
|
||||
- System modules (`edgeAgent`, `edgeHub`) are defined once
|
||||
- Runtime registry credentials are shared across manifests
|
||||
|
||||
## Example: Adding a New Module
|
||||
|
||||
```json
|
||||
{
|
||||
"modulesContent": {
|
||||
"$edgeAgent": {
|
||||
"properties.desired.modules.mynewmodule": {
|
||||
"version": "1.0",
|
||||
"type": "docker",
|
||||
"status": "running",
|
||||
"restartPolicy": "always",
|
||||
"startupOrder": 5,
|
||||
"settings": {
|
||||
"image": "${MODULES.mynewmodule}",
|
||||
"createOptions": {
|
||||
"HostConfig": {
|
||||
"LogConfig": {
|
||||
"Type": "json-file",
|
||||
"Config": {
|
||||
"max-size": "10m",
|
||||
"max-file": "10"
|
||||
}
|
||||
},
|
||||
"Mounts": [
|
||||
{
|
||||
"Type": "volume",
|
||||
"Target": "/app/data/",
|
||||
"Source": "mynewmodule"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"$edgeHub": {
|
||||
"properties.desired.routes.mynewmoduleToIoTHub": {
|
||||
"route": "FROM /messages/modules/mynewmodule/outputs/* INTO $upstream",
|
||||
"priority": 0,
|
||||
"timeToLiveSecs": 86400
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
292
skills/iot-edge-module/references/module-structure.md
Normal file
292
skills/iot-edge-module/references/module-structure.md
Normal file
@@ -0,0 +1,292 @@
|
||||
# IoT Edge Module Structure Reference
|
||||
|
||||
This document describes the standard structure and files for IoT Edge modules in this project.
|
||||
|
||||
## Module Directory Structure
|
||||
|
||||
Each module follows this standard structure:
|
||||
|
||||
```
|
||||
<modulename>/ # Lowercase with "module" suffix
|
||||
├── .dockerignore # Docker build exclusions
|
||||
├── .gitignore # Git exclusions (bin/, obj/)
|
||||
├── Dockerfile.amd64 # Production build
|
||||
├── Dockerfile.amd64.debug # Debug build with vsdbg
|
||||
├── module.json # IoT Edge module metadata
|
||||
├── <ModuleName>.csproj # .NET 9.0 project file
|
||||
├── Program.cs # Application entry point
|
||||
├── GlobalUsings.cs # Global namespace imports
|
||||
├── LoggingEventIdConstants.cs # Logging event IDs
|
||||
├── <ModuleName>Service.cs # Main hosted service
|
||||
├── <ModuleName>ServiceLoggerMessages.cs # Logging messages
|
||||
├── Properties/
|
||||
│ └── launchSettings.json # Local debugging configuration
|
||||
├── Services/ # Optional: Business logic services
|
||||
├── Contracts/ # Optional: Module-specific contracts
|
||||
├── Options/ # Optional: Configuration option classes
|
||||
├── Jobs/ # Optional: Quartz scheduler jobs
|
||||
└── [Other domain-specific folders]
|
||||
```
|
||||
|
||||
## Required Files
|
||||
|
||||
### 1. module.json
|
||||
|
||||
**Purpose**: IoT Edge module metadata for build and deployment.
|
||||
|
||||
**Location**: `<modulename>/module.json`
|
||||
|
||||
**Schema**:
|
||||
```json
|
||||
{
|
||||
"$schema-version": "0.0.1",
|
||||
"description": "Module description",
|
||||
"image": {
|
||||
"repository": "yourregistry.azurecr.io/<modulename>",
|
||||
"tag": {
|
||||
"version": "0.0.${BUILD_BUILDID}",
|
||||
"platforms": {
|
||||
"amd64": "./Dockerfile.amd64",
|
||||
"amd64.debug": "./Dockerfile.amd64.debug"
|
||||
}
|
||||
},
|
||||
"buildOptions": [],
|
||||
"contextPath": "../../../"
|
||||
},
|
||||
"language": "csharp"
|
||||
}
|
||||
```
|
||||
|
||||
**Key fields**:
|
||||
- `repository`: Azure Container Registry URL + lowercase module name
|
||||
- `version`: Uses `${BUILD_BUILDID}` for CI/CD builds
|
||||
- `platforms`: Maps platform to Dockerfile
|
||||
- `contextPath`: Points to repo root (`../../../`) for multi-project Docker builds
|
||||
|
||||
### 2. .csproj
|
||||
|
||||
**Purpose**: .NET project configuration.
|
||||
|
||||
**Target framework**: `net9.0`
|
||||
**Output type**: `Exe` (console application)
|
||||
**Docker target OS**: `Linux`
|
||||
|
||||
**Required dependencies**:
|
||||
- `Atc` - Common utilities
|
||||
- `Atc.Azure.IoTEdge` - IoT Edge abstractions
|
||||
- `Microsoft.Azure.Devices.Client` - IoT Hub SDK
|
||||
- `Microsoft.Extensions.Hosting` - Generic Host
|
||||
|
||||
**Optional project reference**:
|
||||
- Shared contracts project (e.g., `Company.ProjectName.Modules.Contracts`) - Shared constants and contracts across modules
|
||||
|
||||
### 3. Program.cs
|
||||
|
||||
**Purpose**: Application entry point using .NET Generic Host.
|
||||
|
||||
**Standard pattern**:
|
||||
```csharp
|
||||
using var host = Host.CreateDefaultBuilder(args)
|
||||
.ConfigureServices((hostContext, services) =>
|
||||
{
|
||||
services.AddLogging(builder =>
|
||||
{
|
||||
builder.AddModuleConsoleLogging();
|
||||
});
|
||||
|
||||
if (hostContext.IsStandaloneMode())
|
||||
{
|
||||
services.AddSingleton<IModuleClientWrapper, MockModuleClientWrapper>();
|
||||
}
|
||||
else
|
||||
{
|
||||
services.AddModuleClientWrapper(TransportSettingsFactory.BuildMqttTransportSettings());
|
||||
}
|
||||
|
||||
services.AddSingleton<IMethodResponseFactory, MethodResponseFactory>();
|
||||
|
||||
// Add your service registrations here
|
||||
|
||||
services.AddHostedService<YourModuleService>();
|
||||
})
|
||||
.UseConsoleLifetime()
|
||||
.Build();
|
||||
|
||||
await host.RunAsync();
|
||||
```
|
||||
|
||||
**Key components**:
|
||||
- `AddModuleConsoleLogging()` - Structured console logging
|
||||
- `IsStandaloneMode()` - Detects local vs. edge runtime
|
||||
- `AddModuleClientWrapper()` - IoT Hub connectivity with MQTT
|
||||
- `AddHostedService<>()` - Main service registration
|
||||
|
||||
### 4. Main Service File
|
||||
|
||||
**Purpose**: Main module logic as a `BackgroundService`.
|
||||
|
||||
**Naming**: `<ModuleName>Service.cs`
|
||||
|
||||
**Responsibilities**:
|
||||
- Open IoT Hub connection on startup
|
||||
- Register direct method handlers
|
||||
- Implement core module logic
|
||||
- Handle graceful shutdown
|
||||
|
||||
### 5. GlobalUsings.cs
|
||||
|
||||
**Purpose**: Global namespace imports for cleaner code.
|
||||
|
||||
**Standard imports**:
|
||||
```csharp
|
||||
global using Microsoft.Extensions.DependencyInjection;
|
||||
global using Microsoft.Extensions.Hosting;
|
||||
global using Microsoft.Extensions.Logging;
|
||||
```
|
||||
|
||||
### 6. LoggingEventIdConstants.cs
|
||||
|
||||
**Purpose**: Centralized logging event IDs.
|
||||
|
||||
**Standard event IDs**:
|
||||
- `1000` - ModuleStarting
|
||||
- `1001` - ModuleStarted
|
||||
- `1002` - ModuleStopping
|
||||
- `1003` - ModuleStopped
|
||||
|
||||
### 7. ServiceLoggerMessages.cs
|
||||
|
||||
**Purpose**: Compile-time logging using source generators.
|
||||
|
||||
**Pattern**:
|
||||
```csharp
|
||||
internal static partial class YourModuleServiceLoggerMessages
|
||||
{
|
||||
[LoggerMessage(
|
||||
EventId = LoggingEventIdConstants.ModuleStarting,
|
||||
Level = LogLevel.Information,
|
||||
Message = "Module is starting")]
|
||||
internal static partial void LogModuleStarting(this ILogger logger);
|
||||
}
|
||||
```
|
||||
|
||||
### 8. Dockerfile.amd64
|
||||
|
||||
**Purpose**: Production Docker image build.
|
||||
|
||||
**Multi-stage build**:
|
||||
1. **Build stage**: .NET SDK 9.0, restore dependencies, publish release build
|
||||
2. **Runtime stage**: .NET Runtime 9.0, non-root user, security hardening
|
||||
|
||||
**Security features**:
|
||||
- Non-root user (`moduleuser`, UID 2000)
|
||||
- TPM access group (GID 3000)
|
||||
- Minimal runtime image
|
||||
|
||||
### 9. Dockerfile.amd64.debug
|
||||
|
||||
**Purpose**: Debug Docker image with remote debugging support.
|
||||
|
||||
**Additional features**:
|
||||
- vsdbg debugger installation
|
||||
- Debug build configuration
|
||||
|
||||
### 10. .dockerignore
|
||||
|
||||
**Purpose**: Exclude files from Docker build context.
|
||||
|
||||
**Excludes**: bin/, obj/, .git, .vs, node_modules, etc.
|
||||
|
||||
### 11. .gitignore
|
||||
|
||||
**Purpose**: Exclude build artifacts from Git.
|
||||
|
||||
**Excludes**: bin/, obj/
|
||||
|
||||
### 12. Properties/launchSettings.json
|
||||
|
||||
**Purpose**: Local debugging configuration.
|
||||
|
||||
**Required environment variables**:
|
||||
- `IOTEDGE_MODULEID` - Module identifier
|
||||
- `EdgeHubConnectionString` - Local connection string for standalone mode
|
||||
- `EdgeModuleCACertificateFile` - Certificate file path (can be empty)
|
||||
|
||||
## Shared Contracts
|
||||
|
||||
### Module Constants
|
||||
|
||||
**Location**: `<contracts-project-path>/<ModuleName>/<ModuleName>Constants.cs`
|
||||
|
||||
**Purpose**: Shared constants for module identification and direct methods.
|
||||
|
||||
**Structure**:
|
||||
```csharp
|
||||
namespace Company.ProjectName.Modules.Contracts.<ModuleName>;
|
||||
|
||||
public static class <ModuleName>Constants
|
||||
{
|
||||
public const string ModuleId = "<modulename>";
|
||||
|
||||
// Direct method names
|
||||
public const string DirectMethodExample = "ExampleMethod";
|
||||
}
|
||||
```
|
||||
|
||||
## Naming Conventions
|
||||
|
||||
- **Module directory**: Lowercase with "module" suffix (e.g., `mynewmodule`)
|
||||
- **C# classes**: PascalCase without "module" suffix (e.g., `MyNewModule`)
|
||||
- **Namespace**: PascalCase matching class name (e.g., `namespace MyNewModule;`)
|
||||
- **Constants file**: `<ModuleName>Constants.cs` in shared contracts
|
||||
- **Dockerfile**: `Dockerfile.amd64` and `Dockerfile.amd64.debug`
|
||||
|
||||
## Configuration Pattern
|
||||
|
||||
Modules use `IOptions<T>` for configuration:
|
||||
|
||||
1. **Define options class**:
|
||||
```csharp
|
||||
public class MyModuleOptions
|
||||
{
|
||||
public string Setting { get; set; }
|
||||
}
|
||||
```
|
||||
|
||||
2. **Register in DI**:
|
||||
```csharp
|
||||
services.Configure<MyModuleOptions>(hostContext.Configuration);
|
||||
```
|
||||
|
||||
3. **Inject options**:
|
||||
```csharp
|
||||
public MyService(IOptions<MyModuleOptions> options)
|
||||
```
|
||||
|
||||
4. **Set via environment variables** in deployment manifest:
|
||||
```json
|
||||
"env": {
|
||||
"MyModuleOptions__Setting": {
|
||||
"value": "value"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Optional Folders
|
||||
|
||||
- `Services/` - Business logic and integration services
|
||||
- `Contracts/` - Module-specific data contracts (not shared)
|
||||
- `Options/` - Configuration option classes
|
||||
- `Jobs/` - Quartz scheduler job definitions (requires `AddQuartz()`)
|
||||
- `Filters/` - Domain-specific filtering logic
|
||||
- `Providers/` - Factory patterns, client providers
|
||||
- `Publishers/` - Message publishers
|
||||
- `Scrapers/` - Data scraping logic
|
||||
|
||||
## README.md Documentation
|
||||
|
||||
When creating a new module, update `README.md` in the repository root:
|
||||
|
||||
**Section**: "Solution project overview for IoTEdge modules"
|
||||
|
||||
Add your module to the list with a brief description of its purpose.
|
||||
Reference in New Issue
Block a user