Escolar Documentos
Profissional Documentos
Cultura Documentos
Session 510
© 2017 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.
Agenda
Input to Sepia
Contrast
ConcatenatedHue Working Space
O
FilterProgram
Filter
Filter to Output
Performance
Summary of What’s New
Our goal for this release is to enable developers to get better:
Performance Information
Summary of What’s New
Our goal for this release is to enable developers to get better:
11
196 Built-In Filters
ColorBurnBlendMode DisparityToDepth HueSaturationValueGradient PageCurlTransition
AccordionFoldTransition SoftLightBlendMode
ColorClamp DisplacementDistortion Kaleidoscope PageCurlWithShadowTransition
AdditionCompositing SourceAtopCompositing
ColorControls DissolveTransition LabDeltaE ParallelogramTile
AffineClamp SourceInCompositing
ColorCrossPolynomial DivideBlendMode LanczosScaleTransform PDF417BarcodeGenerator
AffineTile SourceOutCompositing
ColorCube DotScreen LenticularHaloGenerator PerspectiveCorrection
AffineTransform SourceOverCompositing
ColorCubesMixedWithMask Droste LightenBlendMode PerspectiveTile
AreaAverage SpotColor
ColorCubeWithColorSpace EdgePreserveUpsampleFilter LightTunnel PerspectiveTransform
AreaHistogram SpotLight
ColorCurves Edges LinearBurnBlendMode PerspectiveTransformWithExtent
AreaMaximum SRGBToneCurveToLinear
ColorDodgeBlendMode EdgeWork LinearDodgeBlendMode PhotoEffectChrome
AreaMaximumAlpha StarShineGenerator
ColorInvert EightfoldReflectedTile LinearGradient PhotoEffectFade
AreaMinimum StraightenFilter
ColorMap ExclusionBlendMode LinearToSRGBToneCurve PhotoEffectInstant
AreaMinimumAlpha StretchCrop
ColorMatrix ExposureAdjust LineOverlay PhotoEffectMono
AreaMinMaxRed StripesGenerator
ColorMonochrome FalseColor LineScreen PhotoEffectNoir
AttributedTextImageGenerator SubtractBlendMode
ColorPolynomial FlashTransition LuminosityBlendMode PhotoEffectProcess
AztecCodeGenerator SunbeamsGenerator
ColorPosterize FourfoldReflectedTile MaskedVariableBlur PhotoEffectTonal
BarcodeGenerator SwipeTransition
ColumnAverage FourfoldRotatedTile MaskToAlpha PhotoEffectTransfer
BarsSwipeTransition TemperatureAndTint
ComicEffect FourfoldTranslatedTile MaximumComponent PinchDistortion
BicubicScaleTransform TextImageGenerator
ConstantColorGenerator GammaAdjust MaximumCompositing PinLightBlendMode
BlendWithAlphaMask Thermal
Convolution3X3 GaussianBlur MedianFilter Pixellate
BlendWithMask ToneCurve
Convolution5X5 GaussianGradient MinimumComponent Pointillize
Bloom TorusLensDistortion
Convolution7X7 GlassDistortion MinimumCompositing QRCodeGenerator
BokehBlur TriangleKaleidoscope
Convolution9Horizontal GlassLozenge ModTransition RadialGradient
BoxBlur TriangleTile
Convolution9Vertical GlideReflectedTile MorphologyGradient RandomGenerator
BumpDistortion TwelvefoldReflectedTile
CopyMachineTransition Gloom MorphologyMaximum RippleTransition
BumpDistortionLinear TwirlDistortion
Crop HardLightBlendMode MorphologyMinimum RowAverage
CheckerboardGenerator UnsharpMask
Crystallize HatchedScreen MotionBlur SaturationBlendMode
CircleSplashDistortion Vibrance
DarkenBlendMode HeightFieldFromMask MultiplyBlendMode ScreenBlendMode
CircularScreen Vignette
DepthBlurEffect HexagonalPixellate MultiplyCompositing SepiaTone
CircularWrap VignetteEffect
DepthOfField HighlightShadowAdjust NinePartStretched ShadedMaterial
Clamp VortexDistortion
DepthToDisparity HistogramDisplayFilter NinePartTiled SharpenLuminance
CMYKHalftone WhitePointAdjust
DifferenceBlendMode HoleDistortion NoiseReduction SixfoldReflectedTile
Code128BarcodeGenerator XRay
DiscBlur HueAdjust OpTile SixfoldRotatedTile
ColorBlendMode ZoomBlur
DisintegrateWithMaskTransition HueBlendMode OverlayBlendMode SmoothLinearGradient
11
New Built-In CIFilters
Some that are useful for Depth filtering
CIDepthToDisparity / CIDisparityToDepth
CIColorCubesMixedWithMask
CIAreaMinMaxRed
New Built-In CIFilters
Some that are useful for Depth filtering
CIDepthToDisparity / CIDisparityToDepth
CIColorCubesMixedWithMask
CIAreaMinMaxRed
CIDepthBlurEffect
New Built-In CIFilters
Some that are useful for Depth filtering
CIDepthToDisparity / CIDisparityToDepth
CIColorCubesMixedWithMask
CIAreaMinMaxRed
CIDepthBlurEffect
CITextImageGenerator
CIColorCurves
CILabDeltaE
CIBicubicScaleTransform
CIBarcodeGenerator
New Built-in CIFilters
Some that are improved
CIHueBlendMode
CISaturationBlendMode
CIColorBlendMode
CILuminosityBlendMode
CILinearBurnBlendMode
New Built-in CIFilters
Some that are improved
CIHueBlendMode
CISaturationBlendMode
CIColorBlendMode
CILuminosityBlendMode
CILinearBurnBlendMode
The demosaic and noise reduction filters used for RAW files
Writing CIKernels in Metal
•
Translate CIKernels
Concatenate CIKernels
Compile to GPU Code
Compiling CIKernels on First Render
Translate CIKernels
Concatenate CIKernels
Compile to GPU Code
Compiling CIKernels on First Render
Render
At run-time
Translate CIKernels
Concatenate CIKernels
Compile to GPU Code
Writing CIKernels in Metal NEW
Benefits
Writing CIKernels in Metal NEW
Benefits
• Precompiled at build-time with error diagnostics
Writing CIKernels in Metal NEW
Benefits
• Precompiled at build-time with error diagnostics
Benefits
• Precompiled at build-time with error diagnostics
Benefits
• Precompiled at build-time with error diagnostics
Benefits
• Precompiled at build-time with error diagnostics
New header file containing CIKernel extensions to the Metal Shading Language
CIKernel Metal Library NEW
New header file containing CIKernel extensions to the Metal Shading Language
• CIKernel data types
CIKernel Metal Library NEW
New header file containing CIKernel extensions to the Metal Shading Language
• CIKernel data types
destination, sampler, sample_t
CIKernel Metal Library NEW
New header file containing CIKernel extensions to the Metal Shading Language
• CIKernel data types
destination, sampler, sample_t
• CIKernel functions
CIKernel Metal Library NEW
New header file containing CIKernel extensions to the Metal Shading Language
• CIKernel data types
destination, sampler, sample_t
• CIKernel functions
premultiply, unpremultiply, srgb_to_linear, linear_to_srgb, compare, cossin, sincos
// CIKernelMetalLib.h
namespace coreimage {
struct destination {
float2 coord() const;
};
struct sampler {
float2 transform(float2 p) const;
float2 coord() const;
float4 sample(float2 p) const;
float4 extent() const;
};
// ...
}
// CIKernelMetalLib.h
namespace coreimage {
struct destination {
float2 coord() const;
};
struct sampler {
float2 transform(float2 p) const;
float2 coord() const;
float4 sample(float2 p) const;
float4 extent() const;
};
// ...
}
// CIKernelMetalLib.h
namespace coreimage {
struct destination {
float2 coord() const;
};
struct sampler {
float2 transform(float2 p) const;
float2 coord() const;
float4 sample(float2 p) const;
float4 extent() const;
};
// ...
}
Destination and Samplers
#include <metal_stdlib>
using namespace metal;
}}
// Metal CIWarpKernel Source
#include <metal_stdlib>
using namespace metal;
}}
// Metal CIWarpKernel Source
#include <metal_stdlib>
using namespace metal;
}}
// Metal CIWarpKernel Source
#include <metal_stdlib>
using namespace metal;
}}
// Metal CIWarpKernel Source
#include <metal_stdlib>
using namespace metal;
}}
// Metal CIColorKernel Source
#include <metal_stdlib>
using namespace metal;
}}
// Metal CIColorKernel Source
#include <metal_stdlib>
using namespace metal;
}}
// Metal CIColorKernel Source
#include <metal_stdlib>
using namespace metal;
}}
// Metal CIKernel Source
#include <metal_stdlib>
using namespace metal;
}}
// Metal CIKernel Source
#include <metal_stdlib>
using namespace metal;
}}
// Metal CIKernel Source
#include <metal_stdlib>
using namespace metal;
}}
How to Create Metal CIKernels
Options: -fcikernel
Compiling and Linking CIKernels
• CVPixelBuffer
• Metal Texture
• OpenGL Texture
• Memory buffer
New CIRenderDestination API
• CVPixelBuffer
• Metal Texture
• OpenGL Texture
• Memory buffer
• Destination colorspace
New CIRenderDestination API
• Destination colorspace
• Blending
New CIRenderDestination API
• Destination colorspace
• Blending
dest.blendKernel = blend // use one of 37 built-in blend kernels or create your own
dest.blendKernel = blend // use one of 37 built-in blend kernels or create your own
dest.blendKernel = blend // use one of 37 built-in blend kernels or create your own
dest.blendKernel = blend // use one of 37 built-in blend kernels or create your own
dest.blendKernel = blend // use one of 37 built-in blend kernels or create your own
dest.blendKernel = blend // use one of 37 built-in blend kernels or create your own
while stillDrawing {
let texture = currentDrawable.texture
CPU
GPU
Rendering to Metal Drawable Textures
CPU
GPU
Get drawable
Rendering to Metal Drawable Textures
CPU
GPU
Get drawable
Call startTask
Rendering to Metal Drawable Textures
Frame A
CPU I1 I2 D
Frame A
GPU
I1 I2 D
Get drawable
Call startTask
Rendering to Metal Drawable Textures
Frame A
CPU I1 I2 D
Frame A
GPU
I1 I2 D
Get drawable
Call startTask
startTask returns
Rendering to Metal Drawable Textures
Frame A
CPU I1 I2 D
Frame A
GPU
I1 I2 D
startTask returns
Rendering to Metal Drawable Textures
Frame A
CPU I1 I2 D
Frame A
GPU
I1 I2 D
startTask returns
Rendering to Metal Drawable Textures
Frame A Frame B
CPU I1 I2 D I1 I2 D
Frame A Frame B
GPU
I1 I2 D I1 I2 D
startTask returns
Rendering to Metal Drawable Textures
startTask returns
Rendering to Metal Drawable Textures Improved
while stillDrawing {
let dest = CIRenderDestination(
width: 1024,
height: 768,
pixelFormat: MTLPixelFormat.rgba8Unorm,
commandBuffer: cmdBuffer) { () -> MTLTexture in
return currentDrawable.texture
})
while stillDrawing {
let dest = CIRenderDestination(
width: 1024,
height: 768,
pixelFormat: MTLPixelFormat.rgba8Unorm,
commandBuffer: cmdBuffer) { () -> MTLTexture in
return currentDrawable.texture
})
while stillDrawing {
let dest = CIRenderDestination(
width: 1024,
height: 768,
pixelFormat: MTLPixelFormat.rgba8Unorm,
commandBuffer: cmdBuffer) { () -> MTLTexture in
return currentDrawable.texture
})
while stillDrawing {
let dest = CIRenderDestination(
width: 1024,
height: 768,
pixelFormat: MTLPixelFormat.rgba8Unorm,
commandBuffer: cmdBuffer) { () -> MTLTexture in
return currentDrawable.texture
})
45
Detailed Image and Render Information
Awesome Core Image Quick Looks in Xcode
46
Detailed Image and Render Information
Awesome Core Image Quick Looks in Xcode
46
Detailed Image and Render Information
Awesome Core Image Quick Looks in Xcode
46
Detailed Image and Render Information
Awesome Core Image Quick Looks in Xcode
46
CIImage Quick Look
// downsample by 2x
image = image.applying(CGAffineTransform(scaleX:0.5, y:0.5))
CIImage Quick Look
// downsample by 2x
image = image.applying(CGAffineTransform(scaleX:0.5, y:0.5))
(CIImage)0x0000000100e61630
CIImage Quick Look
// downsample by 2x
image = image.applying(CGAffineTransform(scaleX:0.5, y:0.5))
(CIImage)0x0000000100e61630
CIImage <0x100e61630>
affine
0.5 0 0
0 0.5 0
extent=[0 0 1512 2016]
opaque
affine
0 -1 0
1 0 4032
extent=[0 0 3024 4032]
opaque
affine
1 0 0
0 -1 3024
extent=[0 0 4032 3024]
opaque
colormatch
Display P3_to_workingspace
extent=[0 0 4032 3024]
opaque
affine
0.5 0 0
0 0.5 0
extent=[0 0 1512 2016]
opaque
affine
0 -1 0
1 0 4032
extent=[0 0 3024 4032]
opaque
affine
1 0 0
0 -1 3024
extent=[0 0 4032 3024]
opaque
colormatch
Display P3_to_workingspace
extent=[0 0 4032 3024]
opaque
affine
affine
0 -1 0
1 01 00 4032
0 -1 30240 3024 4032]
extent=[0
opaque
extent=[0 0 4032 3024]
opaque
affine
1 0 0
0 -1 3024
extent=[0 0 4032 3024]
opaque
colormatch
Display P3_to_workingspace
extent=[0 0 4032 3024]
opaque
colormatch
Display P3_to_workingspace
extent=[0 0 4032 3024]
opaque
IOSurface IOSurface
0x100f483d0(418) seed:1 YCC420f 601 alpha_one
0x100f483d0(189) seed:1 YCC420f 601 alpha_one
extent=[0 extent=[0
0 40320 3024]
4032 3024]
opaque
opaque
opaque
CIImage <0x100e61630>
affine
affine
0 -10.5 00 0
1 00 0.540320
extent=[0 0 1512 2016]
extent=[0
opaque 0 3024 4032]
opaque
affine
0 -1 0
1 0 4032
extent=[0 0 3024 4032]
affine
opaque
1 0 0
0 -1 3024
extent=[0 0 4032 3024]
affine
opaque
1 0 0
0 -1 3024
extent=[0 0 4032 3024]
opaque
colormatch
colormatch
Display P3_to_workingspace
Display P3_to_workingspace
extent=[0 0 4032 3024]
extent=[0 0 4032 3024]
opaque
opaque
affine
0 -1 0
1 0 4032
affine
affine
0 -1
0 -1 0 0 0 3024 4032]
extent=[0
1 01 04032
opaque 4032
affine
extent=[0 0 3024 4032]
extent=[0 0 3024 4032]
0 -1 0
opaque
opaque
1 0 4032
extent=[0 0 3024 4032]
opaque
affine
affine
1 0 0
1 00 -10 3024
affine
0 -1 30240 4032 3024]
extent=[0
opaque
1 0 0
extent=[0 0 4032 3024]
0 -1 3024
opaque
extent=[0 0 4032 3024]
affine
1opaque
0 0
colormatch
0 Display
-1 3024P3_to_workingspace
extent=[0 0 4032 3024]
extent=[0
opaque
0 4032 3024]
opaque
colormatch
Display P3_to_workingspace
colormatch0 4032 3024]
extent=[0
Display
IOSurface opaque P3_to_workingspace
0x100f483d0(189) seed:1 YCC420f 601 alpha_one
extent=[0 extent=[0
0 4032 3024] 0 4032 3024]
opaque
opaque
colormatch
Display P3_to_workingspace
extent=[0 0 4032 3024]
IOSurface 0x1017ac560(276) seed:1 YCC420f 601 alpha_one
CIImage Quick Look
// load disparity image and apply orientation
var disparity = CIImage(
contentsOf: url,
options: [kCIImageAuxiliaryDisparity: true,
kCIImageApplyOrientationProperty: true])
// cubic upsample by 2x
disparity = disparity.applyingFilter(“CIBicubicScaleTransform”, withInputParameters: […])
// cubic upsample by 2x
disparity = disparity.applyingFilter(“CIBicubicScaleTransform”, withInputParameters: […])
(CIImage) 0x0000000100f29480
// adjust disparity mask and blend foreground and background
let mask = disparity.applyingFilter(“CIColorControls",
withInputParameters: [kCIInputContrastKey: 2.0])
mask = mask.applyingFilter(“CIColorClamp")
CIImage Quick Look
// load disparity image and apply orientation
var disparity = CIImage(
contentsOf: url,
options: [kCIImageAuxiliaryDisparity: true,
kCIImageApplyOrientationProperty: true])
// cubic upsample by 2x
disparity = disparity.applyingFilter(“CIBicubicScaleTransform”, withInputParameters: […])
(CIImage) 0x0000000100f29480
// adjust disparity mask and blend foreground and background CIImage <0x100f29480>
mask = mask.applyingFilter(“CIColorClamp")
opaque
affine
1 0 0
0 -1 576
extent=[0 0 768 576]
opaque
colormatch
Linear Gray_to_workingspace
extent=[0 0 768 576]
opaque
kernel _cubicUpsample
src
scale=[0.380952 0.380952 0 0]
coefsLT1=[1.5 -2.5 0 1]
coefsLT2=[-0.5 2.5 -4 2]
extent=[-6 -6 1524 2028]
affine
0 -1 0
1 0 768
extent=[0 0 576 768]
opaque
affine
1 0 0
0 -1 576
extent=[0 0 768 576]
opaque
colormatch
Linear Gray_to_workingspace
extent=[0 0 768 576]
opaque
affine
0 -1 0
1 0 768
kernel _cubicUpsample
extent=[0
src 0 576 768]
opaque
scale=[0.380952 0.380952 0 0]
coefsLT1=[1.5 -2.5 0 1]
coefsLT2=[-0.5 2.5 -4 2]
extent=[-6 -6 1524 2028]
affine
affine
0 -1 0
1 01 00 768
0 -1 5760 576 768]
extent=[0
opaque
extent=[0 0 768 576]
opaque
affine
1 0 0
0 -1 576
extent=[0 0 768 576]
opaque
colormatch
Linear Gray_to_workingspace
extent=[0 0 768 576]
opaque
colormatch
Linear Gray_to_workingspace
extent=[0 0 768 576]
opaque
kernel _cubicUpsample
src kernel _cubicUpsample
src
scale=[0.380952 0.380952
scale=[0.380952 0.380952 0 0] 0 0]
coefsLT1=[1.5 -2.5 0 1]
coefsLT1=[1.5 -2.5 0 1]
coefsLT2=[-0.5 2.5 -4 2]
coefsLT2=[-0.5 2.52028]
extent=[-6 -6 1524 -4 2]
extent=[-6 -6 1524 2028]
affine
0 -1 0
1 0 768
extent=[0 0 576 768]
affine
opaque
0 -1 0
1 0 768
extent=[0 0 576 768]
affine
opaque
1 0 0
0 -1 576
extent=[0 0 768 576]
opaque
affine
colormatch
1Linear
0 Gray_to_workingspace
0
0extent=[0
-1 576 0 768 576]
opaque
extent=[0 0 768 576]
opaque
colormatch
Linear Gray_to_workingspace
extent=[0 0 768 576]
CIImage <0x100f29480>
kernel _cubicUpsample
src
scale=[0.380952 0.380952 0 0]
coefsLT1=[1.5 -2.5 0 1]
coefsLT2=[-0.5 2.5 -4 2]
extent=[-6 -6 1524 2028]
affine
0 -1 0
1 0 768
extent=[0 0 576 768]
opaque
affine
1 0 0
0 -1 576
extent=[0 0 768 576]
opaque
colormatch
Linear Gray_to_workingspace
extent=[0 0 768 576]
opaque
colorkernel _blendWithMask
f=image_0
0 1
color_matrix color_matrix
r=(1.51572 0 0 0) r=(0.659754 0 0 0)
(CIImage) 0x0000000100e341d0
2
g=(0 1.51572 0 0) g=(0 0.659754 0 0)
colorkernel _colorClamp
colormatch colormatch
c
kCIInputMaskImageKey: mask ])
devicergb_to_workingspace devicergb_to_workingspace
lo=[0 0 0 0]
extent=[0 0 1512 2016] extent=[0 0 1512 2016]
hi=[1 1 1 1]
opaque opaque
extent=[-6 -6 1524 2028]
color_matrix
3 4 5 6 r=(2 0 0 0)
g=(0 2 0 0)
colormatch provider BGRA8 32x1024 colormatch provider BGRA8 32x1024
b=(0 0 2 0)
workingspace_to_devicergb alpha_one edge_clamp cache workingspace_to_devicergb alpha_one edge_clamp cache
a=(0 0 0 1)
extent=[0 0 1512 2016] extent=[infinite][0 0 32 1024] extent=[0 0 1512 2016] extent=[infinite][0 0 32 1024]
bias=(-0.5 -0.5 -0.5 0)
opaque opaque opaque opaque
kernel _cubicUpsample
affine
src
0.5 0 0
scale=[0.380952 0.380952 0 0]
0 0.5 0
coefsLT1=[1.5 -2.5 0 1]
extent=[0 0 1512 2016]
coefsLT2=[-0.5 2.5 -4 2]
opaque
extent=[-6 -6 1524 2028]
affine affine
0 -1 0 0 -1 0
1 0 4032 1 0 768
extent=[0 0 3024 4032] extent=[0 0 576 768]
opaque opaque
affine affine
1 0 0 1 0 0
0 -1 3024 0 -1 576
extent=[0 0 4032 3024] extent=[0 0 768 576]
opaque opaque
colormatch colormatch
Display P3_to_workingspace Linear Gray_to_workingspace
extent=[0 0 4032 3024] extent=[0 0 768 576]
opaque opaque
IOSurface 0x100e0aa10(29) seed:1 YCC420f 601 alpha_one IOSurface 0x100e17c00(213) seed:1 Lh alpha_one
extent=[0 0 4032 3024] extent=[0 0 768 576]
opaque opaque
CIImage <0x10353a570>
let output
colormatch = fg.applyingFilter("CIBlendWithMask",
provider BGRA8 32x1024 colormatch provider BGRA8 32x1024
unpremultiply
workingspace_to_devicergb alpha_one edge_clamp cache workingspace_to_devicergb alpha_one edge_clamp cache
extent=[-6 -6 1524 2028]
withInputParameters: [ kCIInputBackgroundImageKey: bg,
extent=[0 0 1512 2016]
opaque
extent=[infinite][0 0 32 1024]
opaque
extent=[0 0 1512 2016]
opaque
extent=[infinite][0 0 32 1024]
opaque
kCIInputMaskImageKey: mask ])
color_matrix
r=(2 0 0 0)
affine
g=(0 2 0 0)
0.5 0 0
b=(0 0 2 0)
0 0.5 0
a=(0 0 0 1)
extent=[0 0 1512 2016]
bias=(-0.5 -0.5 -0.5 0)
opaque
cube=image_4 cube=image_6
lo=[0 00 00 0]
lo=[0 0]
bg = bg.applyingFilter("CIExposureAdjust", withInputParameters: [kCIInputEVKey : -0.6])
dims=[31 0.0322581 0.03125 0.000976562]
extent=[0 0 1512 2016] extent=[0 0 1512 2016]
0.000976562]
dims=[31 0.0322581 0.03125 0.000976562]
hi=[1 11 11 1]
hi=[1 1]
extent=[-6 -6
extent=[-6 -6 1524
1524 2028]
2028]
opaque opaque
let output
colormatch = fg.applyingFilter("CIBlendWithMask",
provider BGRA8 32x1024 colormatch provider BGRA8
provider BGRA8 32x1024
32x1024
unpremultiply
unpremultiply
workingspace_to_devicergb alpha_one edge_clamp cache workingspace_to_devicergb alpha_one edge_clamp
alpha_one edge_clamp cache
cache
extent=[-6 -6
extent=[-6 -6 1524
1524 2028]
2028]
withInputParameters: [ kCIInputBackgroundImageKey: bg,
extent=[0 0 1512 2016]
opaque
extent=[infinite][0 0 32 1024]
opaque
extent=[0 0 1512 2016]
opaque
extent=[infinite][0 00 32
extent=[infinite][0
opaque
opaque
32 1024]
1024]
kCIInputMaskImageKey: mask ])
color_matrix
color_matrix
r=(2 00 00 0)
r=(2 0)
affine
g=(0 22 00 0)
g=(0 0)
0.5 0 0
b=(0 00 22 0)
b=(0 0)
0 0.5 0
a=(0 00 00 1)
a=(0 1)
extent=[0 0 1512 2016]
bias=(-0.5 -0.5
bias=(-0.5 -0.5 -0.5
-0.5 0)
0)
opaque
extent=[-6 -6
extent=[-6 -6 1524
1524 2028]
2028]
CIImage
CIImage <0x1017c9630>
<0x10353a570>
cube=image_4
cube=image_4 cube=image_6
lo=[0 0 0 0]
bg = bg.applyingFilter("CIExposureAdjust", withInputParameters: [kCIInputEVKey : -0.6])
dims=[31
dims=[31 0.0322581
extent=[0
0.0322581 0.03125
extent=[0 00 1512
0.03125 0.000976562]
1512 2016]
2016]
0.000976562] dims=[31 0.0322581 0.03125 0.000976562]
extent=[0 0 1512 2016]
hi=[1 1 1 1]
2028]
extent=[-6 -6 1524 2028]
opaque
opaque opaque
let colormatch
output
colormatch = fg.applyingFilter("CIBlendWithMask",
provider
provider BGRA8
BGRA8 32x1024
32x1024 colormatch provider BGRA8 32x1024
unpremultiply
workingspace_to_devicergb
workingspace_to_devicergb alpha_one
alpha_one edge_clamp
edge_clamp cache
cache workingspace_to_devicergb alpha_one edge_clamp cache
2028]
extent=[-6 -6 1524 2028]
withInputParameters: [ kCIInputBackgroundImageKey: bg,
extent=[0
extent=[0 00 1512
opaque
opaque
1512 2016]
2016] extent=[infinite][0
extent=[infinite][0 00 32
opaque
opaque
32 1024]
1024] extent=[0 0 1512 2016]
opaque
extent=[infinite][0 0 32 1024]
opaque
kCIInputMaskImageKey: mask ])
color_matrix
r=(2 0 0 0)
affine
affine
g=(0 2 0 0)
0.5 00 00
0.5
b=(0 0 2 0)
00 0.5
0.5 00
a=(0 0 0 1)
extent=[0 00 1512
extent=[0 1512 2016]
2016]
bias=(-0.5 -0.5 -0.5 0)
0)
opaque
opaque
b=image_1
m=image_2
extent=[0 0 1512 2016]
m=image_2
extent=[0 0 1512 2016]
// adjustment for foreground
0 1
let fg = image.applyingFilter("CIPhotoEffectFade") 2
colormatch colormatch
color_matrix color_matrix
//0 adjustment
r=(1.51572 0 0) for background r=(0.659754 0 0 0)
2
g=(0 1.51572 0 0) colorkernel _colorcubeopaque_denorm g=(0 0.659754
colorkernel 0 0)
_colorcubeopaque_denorm
let bg
b=(0 0 1.51572 0) = im=image_3
image.applyingFilter("CIPhotoEffectNoir") b=(0 0 0.659754 0)
im=image_5
colorkernel _colorClamp
c premultiply
cube=image_4 cube=image_6
a=(0 0 0 1) a=(00.0322581
0 0 1)0.03125 0.000976562] lo=[0 0 0 0] extent=[-6 -6 1524 2028]
bg = bg.applyingFilter("CIExposureAdjust",
extent=[0 0 1512 2016]
withInputParameters: [kCIInputEVKey : -0.6])
dims=[31 0.0322581 0.03125 0.000976562] dims=[31
extent=[0 0 1512 2016]
hi=[1 1 1 1]
extent=[-6 -6 1524 2028]
opaque opaque
extent=[0 0 1512 2016] extent=[0 0 1512 2016]
opaque opaque
// combine foreground and background using mask
3 4 5 6
let output
colormatch = fg.applyingFilter("CIBlendWithMask",
provider BGRA8 32x1024 colormatch provider BGRA8 32x1024
unpremultiply
workingspace_to_devicergb alpha_one edge_clamp cache workingspace_to_devicergb alpha_one edge_clamp cache
extent=[-6 -6 1524 2028]
withInputParameters: [ kCIInputBackgroundImageKey: bg,
extent=[0 0 1512 2016]
opaque
extent=[infinite][0 0 32 1024]
opaque
extent=[0 0 1512 2016]
opaque
extent=[infinite][0 0 32 1024]
opaque
colorkernel _colorClamp
olormatch kCIInputMaskImageKey:
colormatch mask ]) c
evicergb_to_workingspace devicergb_to_workingspace
lo=[0 0 0 0]
xtent=[0 0 1512 2016] extent=[0 0 1512 2016]
color_matrix hi=[1 1 1 1]
paque opaque r=(2 0 0 0)
affine
g=(0 2 0 0)
extent=[-6 -6 1524 2028]
0.5 0 0
b=(0 0 2 0)
0 0.5 0
a=(0 0 0 1)
extent=[0 0 1512 2016]
bias=(-0.5 -0.5 -0.5 0)
opaque
guard
let task = try? context.startTask(toRender: output, to: dest),
let info = try? task.waitUntilCompleted()
else {
// handle render failure
}
CIRenderTask Quick Look
// render output to an IOSurface
let dest = CIRenderDestination(ioSurface: surface);
guard
let task = try? context.startTask(toRender: output, to: dest),
let info = try? task.waitUntilCompleted()
(CIRenderTask) 0x0000000102806b20
else {
// handle render failure
}
optimized graph
render_to_surface
(metal context 1 frame 1)
format=BGRA8 roi=[0 0 1512 2016]
clamp_to_alpha
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=7FF7DA85C4C272F
premultiply
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=C1C4C4605B7A67F8
linear_to_srgb
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
unpremultiply
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=6E8216B0766EE943
0 1 2
colorkernel _colorClamp
c
srgb_to_linear srgb_to_linear
lo=[0 0 0 0]
rois=[0 0 1512 2016] rois=[0 0 1512 2016]
hi=[1 1 1 1]
extent=[infinite][empty] extent=[infinite][empty]
rois=[0 0 1512 2016]
digest=EAD1A08A4B0E2AFF digest=EE71EA0E37C9BACA
extent=[-6 -6 1524 2028]
digest=74289F843E3F4F71
guard
extent=[infinite][empty] extent=[infinite][empty] extent=[-6 -6 1524 2028]
digest=5888021ED6FE2240 digest=59D03685D7B281C8 digest=FCCD5FFF99FAFE12
color_matrix_3x4
colorkernel _colorcubeopaque_denorm colorkernel _colorcubeopaque_denorm
r=[2 0 0]
im=(3) im=(5)
g=[0 2 0]
cube=(4) cube=(6)
4 6
(CIRenderTask) 0x0000000102806b20
3 5 src
provider BGRA8 32x1024 provider BGRA8 32x1024
scale=[0.380952 0.380952 0 0]
premultiply alpha_one edge_clamp premultiply alpha_one edge_clamp
coefsLT1=[1.5 -2.5 0 1]
rois=[0 0 1512 2016] rois=[0 0 32 1024] rois=[0 0 1512 2016] rois=[0 0 32 1024]
coefsLT2=[-0.5 2.5 -4 2]
extent=[infinite][empty] extent=[infinite][0 0 32 1024] extent=[infinite][empty] extent=[infinite][0 0 32 1024]
rois=[0 0 1512 2016]
digest=41DC5A181A04E2FD opaque digest=41DC5A181A04E2FD opaque
extent=[-6 -6 1524 2028]
digest=4EC4D10594168D56 digest=9EA3D447FA8130A0
digest=EE11208C4277FD65
else { linear_to_srgb
rois=[0 0 1512 2016]
extent=[infinite][empty]
digest=7085717B16282FC6
linear_to_srgb
rois=[0 0 1512 2016]
extent=[infinite][empty]
digest=7085717B16282FC6
affine
0 -1 576
-1 -0 768
rois=[-1 -1 578 770]
extent=[0 0 576 768]
opaque
digest=F26AE90AEAE2BD7E
}
digest=F1AC6EB48CC9C297
color_matrix_3x3
r=[1.22486 -0.225095 1.43051e-05]
crop [0 0 768 576]
g=[-0.0420312 1.04215 3.36338e-05]
rois=[-1 -1 770 578]
b=[-0.0196301 -0.0786319 1.09799]
extent=[0 0 768 576]
rois=[0 0 4032 3024]
opaque
extent=[infinite][empty]
digest=9310A62A244DBB44
opaque
digest=56878F059714A71E
curve
gamma=2.39999 swizzle_rrr1
a=0.947998 b=0.052002 c=0.0769958 d=0.0390015 e=0 f=0 rois=[0 0 768 576]
rois=[0 0 4032 3024] extent=[infinite][empty]
extent=[infinite][empty] opaque
opaque digest=FEEF5EFAAB178D1C
digest=1227B74C49153950
color_matrix_3x4
r=[1 0 1.40752]
g=[1 -0.345491 -0.716948] IOSurface 0x0(0) seed:0 Nil alpha_one edge_clamp
b=[1 1.77898 0] rois=[0 0 768 576]
bias=[-0.70652 0.533302 -0.892976] extent=[infinite][empty]
rois=[0 0 4032 3024] opaque
extent=[infinite][empty] digest=3A29A615F9F448A
opaque
digest=23C22371D291549D
affine
color_matrix_rrra
0 -0.5 1512
rois=[-1 -1 770 578]
-0.5 -0 2016
extent=[0 0 768 576]
rois=[0 0 1512 2016]
opaque
extent=[infinite][empty]
digest=B8397104A14159DD
digest=F1AC6EB48CC9C297
color_matrix_3x3
r=[1.22486 -0.225095 1.43051e-05]
crop [0 0 768 576]
g=[-0.0420312 1.04215 3.36338e-05]
rois=[-1 -1 770 578]
b=[-0.0196301 -0.0786319 1.09799]
extent=[0 0 768 576]
rois=[0 0 4032 3024]
opaque
extent=[infinite][empty]
digest=9310A62A244DBB44
opaque
digest=56878F059714A71E
curve
gamma=2.39999 swizzle_rrr1
a=0.947998 b=0.052002 c=0.0769958 d=0.0390015 e=0 f=0 rois=[0 0 768 576]
rois=[0 0 4032 3024] extent=[infinite][empty]
extent=[infinite][empty] opaque
opaque digest=FEEF5EFAAB178D1C
digest=1227B74C49153950
color_matrix_3x4
r=[1 0 1.40752]
g=[1 -0.345491 -0.716948] IOSurface 0x0(0) seed:0 Nil alpha_one edge_clamp
-1 -0 768
color_matrix_rrra
rois=[0 0 1512 2016]
linear_to_srgb rois=[0 0 1512 2016]
linear_to_srgb
rois=[-1
rois=[-1 -1 578
-1 770 578]770]
extent=[infinite][empty]
rois=[0 0 1512 2016] extent=[infinite][empty]
rois=[0 0 1512 2016]
extent=[0
extent=[0 0 576
0 768 576]768]
digest=7085717B16282FC6
extent=[infinite][empty] digest=7085717B16282FC6
extent=[infinite][empty]
opaque
opaque
digest=99F3F681729FA5A4 digest=99F3F681729FA5A4
digest=F26AE90AEAE2BD7E
digest=F61974C92D5B0B9E
affine
affine
cropcolor_matrix_rrra
[0 0 768 576]
00 -0.5
-0.5 1512
1512
rois=[-1
rois=[-1 -1 770
-1 770 578]578]
-0.5 -0
-0.5 -0 2016
2016
extent=[0
extent=[0 0 768
0 768 576]576]
rois=[0 00 1512
rois=[0 1512 2016]
2016]
opaque
opaque
extent=[infinite][empty]
extent=[infinite][empty]
digest=B8397104A14159DD
digest=179BA22AFF932D96
digest=F1AC6EB48CC9C297
digest=7543B6B4D80E35E3
color_matrix_3x3
color_matrix_3x3
r=[1.22486 -0.225095
r=[1.22486 -0.225095 1.43051e-05]
1.43051e-05]
crop [0 0 768 576]
swizzle_rrr1
g=[-0.0420312 1.04215
g=[-0.0420312 1.04215 3.36338e-05]
3.36338e-05]
rois=[-1
rois=[0 0 768-1576]
770 578]
b=[-0.0196301 -0.0786319
b=[-0.0196301 -0.0786319 1.09799]
1.09799]
extent=[0 0 768 576]
extent=[infinite][empty]
rois=[0 00 4032
rois=[0 4032 3024]
3024]
opaque
opaque
extent=[infinite][empty]
extent=[infinite][empty]
digest=9310A62A244DBB44
digest=7476CC0097772826
opaque
opaque
digest=56878F059714A71E
digest=CFB3EC1C5985E7E0
curve
curve
gamma=2.39999
gamma=2.39999 IOSurface 0x0(0)swizzle_rrr1
seed:0 Nil alpha_one edge_clamp
a=0.947998 b=0.052002
a=0.947998 b=0.052002 c=0.0769958
c=0.0769958 d=0.0390015
d=0.0390015 e=0
e=0 f=0
f=0 rois=[0 0 768 576]
rois=[0 0 768 576]
rois=[0 00 4032
rois=[0 4032 3024]
3024] extent=[infinite][empty]
extent=[infinite][empty]
extent=[infinite][empty]
extent=[infinite][empty] opaque opaque
opaque
opaque digest=FEEF5EFAAB178D1C
digest=FF5B1FBE78FF1C9E
digest=1227B74C49153950
digest=6BDBF61A00035E59
color_matrix_3x4
color_matrix_3x4
r=[1 00 1.40752]
r=[1 1.40752]
g=[1 -0.345491
g=[1 -0.345491 -0.716948]
-0.716948] IOSurface 0x0(0) seed:0 Nil alpha_one edge_clamp
-1 -0 768
rois=[0 0 1512 2016] rois=[0 0 1512 2016] opaque
rois=[-1 -1 578 770]
extent=[infinite][empty] extent=[infinite][empty] digest=F26AE90AEAE
extent=[0 0 576 768]
digest=7085717B16282FC6 digest=7085717B16282FC6
opaque
digest=F26AE90AEAE2BD7E
affine
affine
color_matrix_rrra
color_matrix_rrra
0 0-0.5
-0.5 1512
1512
rois=[-1 -1 770 578] rois=[-1 -1 770 57
-0.5
-0.5 -0-0 2016
2016
extent=[0 0 768 576]
rois=[0 0 1512 2016] extent=[0 0 768 57
rois=[0 0 1512 2016] opaque
extent=[infinite][empty] opaque
extent=[infinite][empty] digest=B8397104A14159DD
digest=F1AC6EB48CC9C297 digest=B8397104A14
digest=F1AC6EB48CC9C297
color_matrix_3x3
r=[1.22486 -0.225095 1.43051e-05]
crop [0 0 768 576]
g=[-0.0420312 1.04215 3.36338e-05]
color_matrix_3x3
b=[-0.0196301 -0.0786319 1.09799]
rois=[-1 -1 770 578]
extent=[0 0 768 576]
r=[1.22486
rois=[0 0 -0.225095
4032 3024] 1.43051e-05]
extent=[infinite][empty]
opaque crop [0 0 768 576]
g=[-0.0420312 1.04215 3.36338e-05] digest=9310A62A244DBB44
opaque rois=[-1 -1 770 57
b=[-0.0196301 -0.0786319 1.09799]
digest=56878F059714A71E extent=[0 0 768 57
rois=[0 0 4032 3024]
opaque
extent=[infinite][empty]
digest=9310A62A244
opaque
digest=56878F059714A71E
curve
gamma=2.39999 swizzle_rrr1
a=0.947998 b=0.052002 c=0.0769958 d=0.0390015 e=0 f=0 rois=[0 0 768 576]
rois=[0 0 4032 3024] extent=[infinite][empty]
extent=[infinite][empty] opaque
opaque digest=FEEF5EFAAB178D1C
digest=1227B74C49153950
curve
gamma=2.39999 swizzle_rrr1
a=0.947998 b=0.052002 c=0.0769958 d=0.0390015 e=0 f=0 rois=[0 0 768 576]
rois=[0 0 4032 3024] extent=[infinite][
extent=[infinite][empty]
color_matrix_3x4 opaque
r=[1 0 1.40752]
opaque digest=FEEF5EFAAB1
g=[1 -0.345491 -0.716948] IOSurface 0x0(0) seed:0 Nil alpha_one edge_clamp
optimized graph
render_to_surface
(metal context 1 frame 1)
format=BGRA8 roi=[0 0 1512 2016]
affine
1 0 0
0 -1 2016
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=114BF1F26BC2E310
clamp_to_alpha
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=7FF7DA85C4C272F
premultiply
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=C1C4C4605B7A67F8
linear_to_srgb
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=28A3941A3DEF8ED3
unpremultiply
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=6E8216B0766EE943
optimized graph
render_to_surface
(metal context 1 frame 1)
format=BGRA8 roi=[0 0 1512 2016]
affine
1 0 0
0 -1 2016
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=114BF1F26BC2E310
clamp_to_alpha
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=7FF7DA85C4C272F
premultiply
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=C1C4C4605B7A67F8
linear_to_srgb
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=28A3941A3DEF8ED3
unpremultiply
rois=[0 0 1512 2016]
extent=[0 0 1512 2016]
digest=6E8216B0766EE943
CIRenderInfo Quick Look
// render output to an IOSurface
let dest = CIRenderDestination(ioSurface: surface);
guard
let task = try? context.startTask(toRender: output, to: dest),
let info = try? task.waitUntilCompleted()
else {
// handle render failure
}
CIRenderInfo Quick Look
// render output to an IOSurface
let dest = CIRenderDestination(ioSurface: surface);
guard
let task = try? context.startTask(toRender: output, to: dest),
let info = try? task.waitUntilCompleted()
else { (CIRenderInfo) 0x0000000100e082b0
// handle render failure
}
CIRenderInfo Quick Look
program graph
render_to_surface
(metal context 1 frame 1 tile 1)
format=BGRA8 roi=[0 0 1512 2016]
guard
premultiply
linear_to_srgb
affine [0 -0.5 -0.5 -0 1512 2016]
color_matrix_3x3 r=[1.22486 -0.225095 1.43051e-05] g=[-0.0420312 1.04215 3.36338e-05] b=[-0.0196301 -0.0786319 1.09799]
curve gamma=2.39999 a=0.947998 b=0.052002 c=0.0769958 d=0.0390015 e=0 f=0
color_matrix_3x4 r=[1 0 1.40752] g=[1 -0.345491 -0.716948] b=[1 1.77898 0] bias=[-0.70652 0.533302 -0.892976]
(CIRenderInfo) 0x0000000100e082b0
time=15.05994 ms
else {
3
1 2
}
opaque opaque
digest=EE11208C4277FD65 digest=33944A11D3550468
digest=4EC4D10594168D56 digest=9EA3D447FA8130A0
time=1.52849 ms
program format=RGBAh
color_matrix_rrra
crop [0 0 768 576]
swizzle_rrr1
rois=[-1 -1 770 578]
extent=[0 0 768 576]
opaque
digest=B8397104A14159DD
time=0.06279 ms
3
1 2
0
program format=RGBAh
provider BGRA8 32x1024 provider BGRA8 32x1024
kernel _cubicUpsample src scale=[0.380952 0.380952 0 0] coefsLT1=[1.5 -2.5 0 1] coefsLT2=[-0.5 2.5 -4 2] IOSurface 0x0(0) seed:0 Nil alpha_one edge_clamp
alpha_one edge_clamp alpha_one edge_clamp
affine [0 -1 -1 -0 576 768] rois=[0 0 4032 3024]
rois=[0 0 32 1024] rois=[0 0 32 1024]
rois=[0 0 1512 2016] extent=[infinite][empty]
extent=[infinite][0 0 32 1024] extent=[infinite][0 0 32 1024]
extent=[-6 -6 1524 2028] opaque
opaque opaque
digest=EE11208C4277FD65 digest=33944A11D3550468
digest=4EC4D10594168D56 digest=9EA3D447FA8130A0
time=1.52849 ms
program format=RGBAh
color_matrix_rrra
crop [0 0 768 576]
swizzle_rrr1
rois=[-1 -1 770 578]
extent=[0 0 768 576]
opaque
digest=B8397104A14159DD
time=0.06279 ms
program format=RGBAh
color_matrix_rrra
program format=RGBAh
color_matrix_rrra
crop [0 0 768 576]
crop [0 0 768 576]
swizzle_rrr1
swizzle_rrr1
rois=[-1
rois=[-1-1 770578]
-1 770 578]
extent=[0 0 768 576]
extent=[0 0 768 576]
opaque
opaque
digest=B8397104A14159DD
time=0.06279 ms
digest=B8397104A14159DD
time=0.06279 ms
Code128
Aztec
QRCode
PDF417
Framework Barcode Support
Several frameworks support various barcodes types
CIBarcodeDescriptor
CIBarcodeDescriptor Objects Contain
• QRCode’s maskPattern
// Get a CIBarcodeDescriptor from AVFoundation.framework
Core Image: Make a photo from the median of the aligned frames
Photo from Video with Removal of Unwanted Objects
Core Image: Make a photo from the median of the aligned frames
Photo from Video with Removal of Unwanted Objects
Core Image: Make a photo from the median of the aligned frames
Photo from Video with Removal of Unwanted Objects
Core Image: Make a photo from the median of the aligned frames
// Use Vision to find homographic registration and pass it to CI
float4 medianReduction5(sample_t v0, sample_t v1, sample_t v2, sample_t v3, sample_t v4)
{
// using a Bose-Nelson sorting network
swap(v0, v1); swap(v3, v4); swap(v2, v4); swap(v2, v3); swap(v0, v3);
swap(v0, v2); swap(v1, v4); swap(v1, v3); swap(v1, v2);
return v2;
}
// Core Image Metal kernel to return the median of 5 images
float4 medianReduction5(sample_t v0, sample_t v1, sample_t v2, sample_t v3, sample_t v4)
{
// using a Bose-Nelson sorting network
swap(v0, v1); swap(v3, v4); swap(v2, v4); swap(v2, v3); swap(v0, v3);
swap(v0, v2); swap(v1, v4); swap(v1, v3); swap(v1, v2);
return v2;
}
// Core Image Metal kernel to return the median of 5 images
float4 medianReduction5(sample_t v0, sample_t v1, sample_t v2, sample_t v3, sample_t v4)
{
// using a Bose-Nelson sorting network
swap(v0, v1); swap(v3, v4); swap(v2, v4); swap(v2, v3); swap(v0, v3);
swap(v0, v2); swap(v1, v4); swap(v1, v3); swap(v1, v2);
return v2;
}
Demo
•
Photos Editing and Core Image Lab Technology Lab F Thu 3:10PM–6:00PM