TIBCO Data Virtualization Splunk Adapter Guide

Two-Second Adva

TIBCO Data Virtualization®

Splunk Adapter GuideVersion 8.4

Last Updated: January 25, 2021

ntage®

Important Information SOME TIBCO SOFTWARE EMBEDS OR BUNDLES OTHER TIBCO SOFTWARE. USE OF SUCH EMBEDDED OR BUNDLED TIBCO SOFTWARE IS SOLELY TO ENABLE THE FUNCTIONALITY (OR PROVIDE LIMITED ADD-ON FUNCTIONALITY) OF THE LICENSED TIBCO SOFTWARE. THE EMBEDDED OR BUNDLED SOFTWARE IS NOT LICENSED TO BE USED OR ACCESSED BY ANY OTHER TIBCO SOFTWARE OR FOR ANY OTHER

PURPOSE.

USE OF TIBCO SOFTWARE AND THIS DOCUMENTATION IS SUBJECT TO THE TERMS AND CONDITIONS OF A LICENSE AGREEMENT FOUND IN EITHER A SEPARATELY EXECUTED SOFTWARE LICENSE AGREEMENT, OR, IF THERE IS NO SUCH SEPARATE AGREEMENT, THE CLICKWRAP END USER LICENSE AGREEMENT WHICH IS DISPLAYED DURING DOWNLOAD OR INSTALLATION OF THE SOFTWARE (AND WHICH IS DUPLICATED IN THE LICENSE FILE) OR IF THERE IS NO SUCH SOFTWARE LICENSE AGREEMENT OR CLICKWRAP END USER LICENSE AGREEMENT, THE LICENSE(S) LOCATED IN THE “LICENSE” FILE(S) OF THE SOFTWARE. USE OF THIS DOCUMENTATION IS SUBJECT TO THOSE TERMS AND CONDITIONS, AND YOUR USE HEREOF SHALL CONSTITUTE ACCEPTANCE OF AND AN AGREEMENT TO BE BOUND BY THE SAME.

This document is subject to U.S. and international copyright laws and treaties. No part of this document may be reproduced in any form without the written authorization of TIBCO Software Inc.

TIBCO and the TIBCO logo are either registered trademarks or trademarks of TIBCO Software Inc. in the United

States and/or other countries

TIBCO, Two-Second Advantage, TIBCO Spotfire, TIBCO ActiveSpaces, TIBCO Spotfire Developer, TIBCO EMS,

TIBCO Spotfire Automation Services, TIBCO Enterprise Runtime for R, TIBCO Spotfire Server, TIBCO Spotfire

Web Player, TIBCO Spotfire Statistics Services, S-PLUS, and TIBCO Spotfire S+ are either registered trademarks

or trademarks of TIBCO Software Inc. in the United States and/or other countries.

All other product and company names and marks mentioned in this document are the property of their

respective owners and are mentioned for identification purposes only.

THIS SOFTWARE MAY BE AVAILABLE ON MULTIPLE OPERATING SYSTEMS. HOWEVER, NOT ALL OPERATING SYSTEM PLATFORMS FOR A SPECIFIC SOFTWARE VERSION ARE RELEASED AT THE SAME TIME. SEE THE README FILE FOR THE AVAILABILITY OF THIS SOFTWARE VERSION ON A SPECIFIC OPERATING SYSTEM PLATFORM.

THIS DOCUMENTATION IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT.

THIS DOCUMENTATION COULD INCLUDE TECHNICAL INACCURACIES OR TYPOGRAPHICAL ERRORS. CHANGES ARE PERIODICALLY ADDED TO THE INFORMATION HEREIN; THESE CHANGES WILL BE INCORPORATED IN NEW EDITIONS OF THIS DOCUMENTATION. TIBCO SOFTWARE INC. MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCT(S) AND/OR THE PROGRAM(S) DESCRIBED IN THIS DOCUMENTATION AT ANY TIME.

THE CONTENTS OF THIS DOCUMENTATION MAY BE MODIFIED AND/OR QUALIFIED, DIRECTLY OR INDIRECTLY, BY OTHER DOCUMENTATION WHICH ACCOMPANIES THIS SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY RELEASE NOTES AND "READ ME" FILES.

Copyright © 2002-2021 TIBCO Software Inc. ALL RIGHTS RESERVED.

TIBCO Software Inc. Confidential Information

TIBCO® Data Virtualization

Contents | 1

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2

Product-Specific Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2How to Access TIBCO Documentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3How to Contact TIBCO Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3How to Join TIBCO Community . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

TDV Splunk Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

Getting Started. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Deploying the Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Connecting to Splunk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Basic Tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Advanced Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Connection String Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Configure Logging for the Splunk Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Advanced Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Troubleshooting the Connection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

SQL Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25SELECT Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26SELECT INTO Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28INSERT Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28UPDATE Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29DELETE Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30EXECUTE Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Data Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

|2

Preface

Forinformationonthefollowing,seetheTDVUserGuide:

• AddingaDataSource

• IntrospectingaDataSource

• TestingtheConnectiontoYourDataSource

Documentation for this and other TIBCO products is available on the TIBCO Documentation site. This site is updated more frequently than any documentation that might be included with the product. To ensure that you are accessing the latest available help topics, please visit:

• https://docs.tibco.com

Product-Specific Documentation

The following documents form the TIBCO® Data Virtualization(TDV) documentation set:

• Users

TDV Getting Started Guide

TDV User Guide

TDV Client Interfaces Guide

TDV Tutorial Guide

TDV Northbay Example

• Administration

TDV Installation and Upgrade Guide

TDV Administration Guide

TDV Active Cluster Guide

TDV Security Features Guide

• Data Sources

TDV Adapter Guides

TDV Data Source Toolkit Guide (Formerly Extensibility Guide)


https://docs.tibco.com

Preface |3

• References

TDV Reference Guide

TDV Application Programming Interface Guide

• Other

TDV Business Directory Guide

TDV Discovery Guide

• TIBCO TDV and Business Directory Release Notes Read the release notes for a list of new and changed features. This document also contains lists of known issues and closed issues for this release.

How to Access TIBCO Documentation

Documentation for TIBCO products is available on the TIBCO Product Documentation website mainly in the HTML and PDF formats.

The TIBCO Product Documentation website is updated frequently and is more current than any other documentation included with the product. To access the latest documentation, visit https://docs.tibco.com.

Documentation for TIBCO Data Virtualization is available on https://docs.tibco.com/products/tibco-data-virtualization-server.

How to Contact TIBCO Support

You can contact TIBCO Support in the following ways:

• For an overview of TIBCO Support, visit https://www.tibco.com/services/support.

• For accessing the Support Knowledge Base and getting personalized content about products you are interested in, visit the TIBCO Support portal at https://support.tibco.com.

• For creating a Support case, you must have a valid maintenance or support contract with TIBCO. You also need a user name and password to log in to https://support.tibco.com. If you do not have a user name, you can request one by clicking Register on the website.


https://docs.tibco.com

https://docs.tibco.com/products/tibco-data-virtualization-server

https://www.tibco.com/services/support

https://support.tibco.com

https://support.tibco.com

4 | Preface

How to Join TIBCO Community

TIBCO Community is the official channel for TIBCO customers, partners, and employee subject matter experts to share and access their collective experience. TIBCO Community offers access to Q&A forums, product wikis, and best practices. It also offers access to extensions, adapters, solution accelerators, and tools that extend and enable customers to gain full value from TIBCO products. In addition, users can submit and vote on feature requests from within the TIBCO Ideas Portal. For a free registration, go to https://community.tibco.com.


https://ideas.tibco.com/

https://ideas.tibco.com/

https://community.tibco.com

|5

TDV Splunk Adapter

Splunk Version Support

The adapter enables you to access data models in Splunk Enterprise or Splunk Cloud as relational databases, enabling bidirectional access to reports, datasets, and table datasets.

Requirements and Restrictions

The Supported SQL section shows the SQL syntax supported by the adapter and points out any limitations.

Getting Started

Deploying the Adapter

For instructions on deploying the adapter, refer to the Installation Guide, section Installing the Advanced Adapters.

Connecting to Splunk

Basic Tab shows how to authenticate to Splunk and configure any necessary connection properties. Additional adapter capabilities can be configured using the available Connection properties on the Advanced tab. The Advanced Settings section shows how to set up more advanced configurations and troubleshoot connection errors.

Basic Tab

Authenticating to Splunk

To authenticate requests, set the User, Password, and URL properties to valid Splunk credentials. By default the adapter makes requests on port 8089.

The adapter uses plaintext authentication by default, since the adapter attempts to negotiate TLS/SSL with the server.

See Advanced Settings for more information on TLS/SSL configuration.


6 | Advanced Tab

Advanced Tab

The connection string properties describe the various options that can be used to establish a connection.

Connection String Options

The following is the full list of the options you can configure in the connection string for this provider.

Firewall Password

A password used to authenticate to a proxy-based firewall.

Firewall Port The TCP port for a proxy-based firewall.

Firewall Server

The name or IP address of a proxy-based firewall.

Firewall Type The protocol used by a proxy-based firewall.

Firewall User The user name to use to authenticate with a proxy-based firewall.

Location A path to the directory that contains the schema files defining tables, views, and stored procedures.

Max Rows Limits the number of rows returned rows when no aggregation or group by is used inthe query. This helps avoid performance issues at design time.

Max Threads Specifies the number of concurrent requests. Only used when UseJobs is true.

Other These hidden properties are used only in specific use cases.

Page Size The number of results to return per page of data retrieved from the Splunk server.

Password The password of the user.

Proxy Auth Scheme

The authentication type to use to authenticate to the ProxyServer proxy.

Proxy Auto Detect

This indicates whether to use the system proxy settings or not. This takes precedence over other proxy settings, so you'll need to set ProxyAutoDetect to FALSE in order use custom proxy settings.


Advanced Tab |7

Firewall Password

A password used to authenticate to a proxy-based firewall.

Data Type

string

Proxy Exceptions

A semicolon separated list of hosts or IPs that are exempt from connecting through the ProxyServer .

Proxy Password

A password to be used to authenticate to the ProxyServer proxy.

Proxy Port The TCP port the ProxyServer proxy is running on.

Proxy Server The hostname or IP address of a proxy to route HTTP traffic through.

Proxy SSL Type

The SSL type to use when connecting to the ProxyServer proxy.

Proxy User A user name to be used to authenticate to the ProxyServer proxy.

Readonly You can use this property to enforce read-only access to Splunk from the provider.

Row Scan Depth

Set this property to control the number of rows scanned when TypeDetectionSchemeis set to RowScan.

SSL Server Cert

The certificate to be accepted from the server when connecting using TLS/SSL.

Timeout The value in seconds until the timeout error is thrown, canceling the operation.

Type Detection Scheme

Determines how to determine the data type of columns.

URL The URL to your Splunk endpoint.

Use Client Side Paging

Whether or not the CData ADO.NET Provider for Splunk should use client side paging. This setting is only applied when UseJobs=true.

Use Jobs Specifies whether to use the jobs endpoint instead of the export endpoint. If set to true, the maximum number of returned rows is configured Splunk's limit.conf file.

User The name of a user.


8 | Advanced Tab

Default Value

""

Remarks

This property is passed to the proxy specified by FirewallServer and FirewallPort, following the authentication method specified by FirewallType.

Firewall Port

The TCP port for a proxy-based firewall.

Data Type

string

Default Value

""

Remarks

This specifies the TCP port for a proxy allowing traversal of a firewall. Use FirewallServer to specify the name or IP address. Specify the protocol with FirewallType.

Firewall Server

The name or IP address of a proxy-based firewall.

Data Type

string

Default Value

""

Remarks

This property specifies the IP address, DNS name, or host name of a proxy allowing traversal of a firewall. The protocol is specified by FirewallType: Use FirewallServer with this property to connect through SOCKS or do tunneling. Use ProxyServer to connect to an HTTP proxy.


Advanced Tab |9

Note that the adapter uses the system proxy by default. To use a different proxy, set ProxyAutoDetect to false.

Firewall Type

The protocol used by a proxy-based firewall.

Data Type

string

Default Value

"NONE"

Remarks

This property specifies the protocol that the adapter will use to tunnel traffic through the FirewallServer proxy. Note that by default, the adapter connects to the system proxy; to disable this behavior and connect to one of the following proxy types, set ProxyAutoDetect to false.

To connect to HTTP proxies, use ProxyServer and ProxyPort. To authenticate to HTTP proxies, use ProxyAuthScheme, ProxyUser, and ProxyPassword.

Firewall User

The user name to use to authenticate with a proxy-based firewall.

Type Default Port

Description

TUNNEL

80 When this is set, the adapter opens a connection to Splunk and traffic flows backand forth through the proxy.

SOCKS4 1080 When this is set, the adapter sends data through the SOCKS 4 proxy specified byFirewallServer and FirewallPort and passes the FirewallUser value to the proxy,which determines if the connection request should be granted.

SOCKS5 1080 When this is set, the adapter sends data through the SOCKS 5 proxy specified byFirewallServer and FirewallPort. If your proxy requires authentication, set FirewallUser and FirewallPassword to credentials the proxy recognizes.


10 | Advanced Tab

Data Type

string

Default Value

""

Remarks

The FirewallUser and FirewallPassword properties are used to authenticate against the proxy specified in FirewallServer and FirewallPort, following the authentication method specified in FirewallType.

Location

A path to the directory that contains the schema files defining tables, views, and stored procedures.

Data Type

string

Default Value

"%APPDATA%\\CData\\Splunk Data Provider\\Schema"

Remarks

The path to a directory which contains the schema files for the adapter (.rsd files for tables and views, .rsb files for stored procedures). The folder location can be a relative path from the location of the executable. The Location property is only needed if you want to customize definitions (for example, change a column name, ignore a column, and so on) or extend the data model with new tables, views, or stored procedures.

If left unspecified, the default location is "%APPDATA%\\CData\\Splunk Data Provider\\Schema" with %APPDATA% being set to the user's configuration directory:

Platform %APPDATA%

Windows The value of the APPDATA environment variable


Advanced Tab |11

Max Rows

Limits the number of rows returned rows when no aggregation or group by is used in the query. This helps avoid performance issues at design time.

Data Type

string

Default Value

"-1"

Remarks

Limits the number of rows returned rows when no aggregation or group by is used in the query. This helps avoid performance issues at design time.

Max Threads

Specifies the number of concurrent requests. Only used when UseJobs is true.

Data Type

string

Default Value

"10"

Remarks

This property allows you to issue multiple requests simultaneously, thereby improving performance.

Other

These hidden properties are used only in specific use cases.

Mac ~/Library/Application Support

Linux ~/.config


12 | Advanced Tab

Data Type

string

Default Value

""

Remarks

The properties listed below are available for specific use cases. Normal driver use cases and functionality should not require these properties.

Specify multiple properties in a semicolon-separated list.

Integration and Formatting

Page Size

The number of results to return per page of data retrieved from the Splunk server.

Data Type

string

Default Value

"1000"

Remarks

The number of results to return per page of data retrieved from the Splunk server.

Password

The password of the user.

DefaultColumnSize Sets the default length of string fields when the data source does not provide column length in the metadata. The default value is 2000.

ConvertDateTimeToGMT Determines whether to convert date-time values to GMT, instead of the local time of the machine.

RecordToFile=filename Records the underlying socket data transfer to the specified file.


Advanced Tab |13

Data Type

string

Default Value

""

Remarks

Together with User, this field is used to authenticate against the Splunk server.

Proxy Auth Scheme

The authentication type to use to authenticate to the ProxyServer proxy.

Data Type

string

Default Value

"BASIC"

Remarks

This value specifies the authentication type to use to authenticate to the HTTP proxy specified by ProxyServer and ProxyPort.

Note that the adapter will use the system proxy settings by default, without further configuration needed; if you want to connect to another proxy, you will need to set ProxyAutoDetect to false, in addition to ProxyServer and ProxyPort. To authenticate, set ProxyAuthScheme and set ProxyUser and ProxyPassword, if needed.

The authentication type can be one of the following:

• BASIC: The adapter performs HTTP BASIC authentication.

• DIGEST: The adapter performs HTTP DIGEST authentication.

• NEGOTIATE: The adapter retrieves an NTLM or Kerberos token based on the applicable protocol for authentication.

• PROPRIETARY: The adapter does not generate an NTLM or Kerberos token. You must supply this token in the Authorization header of the HTTP request.

If you need to use another authentication type, such as SOCKS 5 authentication, see Firewall Type.


14 | Advanced Tab

Proxy Auto Detect

This indicates whether to use the system proxy settings or not. This takes precedence over other proxy settings, so you'll need to set ProxyAutoDetect to FALSE in order use custom proxy settings.

Data Type

bool

Default Value

true

Remarks

This takes precedence over other proxy settings, so you'll need to set ProxyAutoDetect to FALSE in order use custom proxy settings.

NOTE: When this property is set to True, the proxy used is determined as follows:

• A search from the JVM properties (http.proxy, https.proxy, socksProxy, etc.) is performed.

• In the case that the JVM properties don't exist, a search from

• java.home/lib/net.properties is performed.

• In the case that java.net.useSystemProxies is set to True, a search from

• the SystemProxy is performed.

• In Windows only, an attempt is made to retrieve these properties from the

• Internet Options in the registry.

To connect to an HTTP proxy, see Proxy Server. For other proxies, such as SOCKS or tunneling, see Firewall Type.

Proxy Exceptions

A semicolon separated list of hosts or IPs that are exempt from connecting through the ProxyServer .

Data Type

string


Advanced Tab |15

Default Value

""

Remarks

The ProxyServer is used for all addresses, except for addresses defined in this property. Use semicolons to separate entries.

Note that the adapter uses the system proxy settings by default, without further configuration needed; if you want to explicitly configure proxy exceptions for this connection, you need to set ProxyAutoDetect = false, and configure ProxyServer and ProxyPort. To authenticate, set ProxyAuthScheme and set ProxyUser and ProxyPassword, if needed.

Proxy Password

A password to be used to authenticate to the ProxyServer proxy.

Data Type

string

Default Value

""

Remarks

This property is used to authenticate to an HTTP proxy server that supports NTLM (Windows), Kerberos, or HTTP authentication. To specify the HTTP proxy, you can set ProxyServer and ProxyPort. To specify the authentication type, set ProxyAuthScheme.

If you are using HTTP authentication, additionally set ProxyUser and ProxyPassword to HTTP proxy.

If you are using NTLM authentication, set ProxyUser and ProxyPassword to your Windows password. You may also need these to complete Kerberos authentication.

For SOCKS 5 authentication or tunneling, see Firewall Type.

By default, the adapter uses the system proxy. If you want to connect to another proxy, set ProxyAutoDetect to false.


16 | Advanced Tab

Proxy Port

The TCP port the ProxyServer proxy is running on.

Data Type

string

Default Value

"80"

Remarks

The port the HTTP proxy is running on that you want to redirect HTTP traffic through. Specify the HTTP proxy in ProxyServer. For other proxy types, see Firewall Type.

Proxy Server

The hostname or IP address of a proxy to route HTTP traffic through.

Data Type

string

Default Value

""

Remarks

The hostname or IP address of a proxy to route HTTP traffic through. The adapter can use the HTTP, Windows (NTLM), or Kerberos authentication types to authenticate to an HTTP proxy.

If you need to connect through a SOCKS proxy or tunnel the connection, see Firewall Type.

By default, the adapter uses the system proxy. If you need to use another proxy, set ProxyAutoDetect to false.

Proxy SSL Type

The SSL type to use when connecting to the ProxyServer proxy.


Advanced Tab |17

Data Type

string

Default Value

"AUTO"

Remarks

This property determines when to use SSL for the connection to an HTTP proxy specified by ProxyServer. This value can be AUTO, ALWAYS, NEVER, or TUNNEL. The applicable values are the following:

Proxy User

A user name to be used to authenticate to the ProxyServer proxy.

Data Type

string

Default Value

""

Remarks

The ProxyUser and ProxyPassword options are used to connect and authenticate against the HTTP proxy specified in ProxyServer.

You can select one of the available authentication types in ProxyAuthScheme. If you are using HTTP authentication, set this to the user name of a user recognized by the HTTP proxy. If you are using Windows or Kerberos authentication, set this property to a user name in one of the following formats: user@domain

AUTO Default setting. If the URL is an HTTPS URL, the adapter will use the TUNNEL option. If the URL is an HTTP URL, the component will use the NEVER option.

ALWAYS The connection is always SSL enabled.

NEVER The connection is not SSL enabled.

TUNNEL The connection is through a tunneling proxy. The proxy server opens a connection to the remote host and traffic flows back and forth through the proxy.


18 | Advanced Tab

domain\user

Readonly

You can use this property to enforce read-only access to Splunk from the provider.

Data Type

bool

Default Value

false

Remarks

If this property is set to true, the adapter will allow only SELECT queries. INSERT, UPDATE, DELETE, and stored procedure queries will cause an error to be thrown.

Row Scan Depth

Set this property to control the number of rows scanned when TypeDetectionScheme is set to RowScan.

Data Type

string

Default Value

"50"

Remarks

Determines the number of rows used to determine the column data types.

Setting a high value may decrease performance. Setting a low value may prevent the data type from being determined properly, especially when there is null data.

SSL Server Cert

The certificate to be accepted from the server when connecting using TLS/SSL.


Advanced Tab |19

Data Type

string

Default Value

""

Remarks

If using a TLS/SSL connection, this property can be used to specify the TLS/SSL certificate to be accepted from the server. Any other certificate that is not trusted by the machine is rejected.

This property can take the following forms:

If not specified, any certificate trusted by the machine is accepted.

Certificates are validated as trusted by the machine based on the System's trust store. The trust store used is the 'javax.net.ssl.trustStore' value specified for the system. If no value is specified for this property, Java's default trust store is used (for example, JAVA_HOME\lib\security\cacerts).

Use '*' to signify to accept all certificates. Note that this is not recommended due to security concerns.

Timeout

The value in seconds until the timeout error is thrown, canceling the operation.

Description Example

A full PEM Certificate (example shortened for brevity)

-----BEGIN CERTIFICATE----- MIIChTCCAe4CAQAwDQYJKoZIhv......Qw== -----END CERTIFICATE-----

A path to a local file containing the certificate C:\cert.cer

The public key (example shortened for brevity)

-----BEGIN RSA PUBLIC KEY----- MIGfMA0GCSq......AQAB -----END RSA PUBLIC KEY-----

The MD5 Thumbprint (hex values can also be either space or colon separated)

ecadbdda5a1529c58a1e9e09828d70e4

The SHA1 Thumbprint (hex values can also be either space or colon separated)

34a929226ae0819f2ec14b4a3d904f801cbb150d


20 | Advanced Tab

Data Type

string

Default Value

"60"

Remarks

If Timeout = 0, operations do not time out. The operations run until they complete successfully or until they encounter an error condition.

If Timeout expires and the operation is not yet complete, the adapter throws an exception.

Type Detection Scheme

Determines how to determine the data type of columns.

Data Type

string

Default Value

"RowScan"

Remarks

URL

The URL to your Splunk endpoint.

Data Type

string

None Setting TypeDetectionScheme to None will return all columns as the string type.

RowScan Setting TypeDetectionScheme to RowScan will scan rows to heuristically determine the data type. The RowScanDepth determines the number of rows to be scanned.


Advanced Tab |21

Default Value

""

Remarks

The URL to your Splunk endpoint; for example, https://yoursitename.splunk.com:8089.

Use Client Side Paging

Whether or not the CData ADO.NET Provider for Splunk should use client side paging. This setting is only applied when UseJobs=true.

Data Type

bool

Default Value

true

Remarks

Setting this value to false will cause only a single thread to be used to execute this operation instead of attempting to use multiple threads to improve performance.

Use Jobs

Specifies whether to use the jobs endpoint instead of the export endpoint. If set to true, the maximum number of returned rows is configured Splunk's limit.conf file.

Data Type

bool

Default Value

false

Remarks

Whether to use the jobs endpoint instead of the export endpoint.


22 | Logging

User

The name of a user.

Data Type

string

Default Value

""

Remarks

Together with Password, this field is used to authenticate against the Splunk server.

Logging

The adapter uses log4j to generate log files. The settings within the log4j configuration file are used by the adapter to determine the type of messages to log. The following categories can be specified:

• Error: Only error messages are logged.

• Info: Both Error and Info messages are logged.

• Debug: Error, Info, and Debug messages are logged.

The Other property of the adapter can be used to set Verbosity to specify the amount of detail to be included in the log file, that is: Verbosity=4;

You can use Verbosity to specify the amount of detail to include in the log within a category. The following verbosity levels are mapped to the log4j categories:

• 0 = Error

• 1-2 = Info

• 3-5 = Debug

For example, if the log4j category is set to DEBUG, the Verbosity option can be set to 3 for the minimum amount of debug information or 5 for the maximum amount of debug information.


Logging |23

Note that the log4j settings override the Verbosity level specified. The adapter never logs at a Verbosity level greater than what is configured in the log4j properties. In addition, if Verbosity is set to a level less than the log4j category configured, Verbosity defaults to the minimum value for that particular category. For example, if Verbosity is set to a value less than 3 and the Debug category is specified, the Verbosity defaults to 3.

The following list is a breakdown of the Verbosity levels and the information that they log.

• 1 - Will log the query, the number of rows returned by it, the start of execution and the time taken, and any errors.

• 2 - Will log everything included in Verbosity 1 and HTTP headers.

• 3 - Will additionally log the body of the HTTP requests.

• 4 - Will additionally log transport-level communication with the data source. This includes SSL negotiation.

• 5 - Will additionally log communication with the data source and additional details that may be helpful in troubleshooting problems. This includes interface commands.

Configure Logging for the Splunk Adapter

By default, logging is turned on without debugging. If debugging information is desired, uncomment the following line in the TDV Server's log4j.properties file (default location of this file is: C:\Program Files\TIBCO\TDV Server <version>\conf\server):

log4j.logger.com.cdata=DEBUG

The TDV Server must be restarted after changing the log4j.properties file, which can be accomplished by running the composite.bat script located at: C:\Program Files\TIBCO\TDV Server <version>\bin. Note that reauthenticating to the TDV Studio is required after restarting the server.

Here is an example of the calls: .\composite.bat monitor restart

All logs for the adapter are written to the "cs_cdata.log" file as specified in the log4j properties.

Note: The "log4j.logger.com.cdata=DEBUG" option is not required if the Debug Output Enabled option is set to true within the TDV Studio. To set this option, navigate to Administrator > Configuration. Select Server > Configuration > Debugging and set the Debug Output Enabled option to True.


24 | Advanced Settings

Advanced Settings

Customizing the SSL Configuration

By default, the adapter attempts to negotiate SSL/TLS by checking the server's certificate against the system's trusted certificate store. To specify another certificate, see the SSL Server Cert property for the available formats to do so.

Connecting Through a Firewall or Proxy

HTTP Proxies

To connect through the Windows system proxy, you do not need to set any additional connection properties. To connect to other proxies, set ProxyAutoDetect to false.

In addition, to authenticate to an HTTP proxy, set ProxyAuthScheme, ProxyUser, and ProxyPassword, in addition to ProxyServer and ProxyPort.

Other Proxies

Set the following properties:

• To use a proxy-based firewall, set FirewallType, FirewallServer, and FirewallPort.

• To tunnel the connection, set FirewallType to TUNNEL.

• To authenticate, specify FirewallUser and FirewallPassword.

• To authenticate to a SOCKS proxy, additionally set FirewallType to SOCKS5.

Troubleshooting the Connection

To show adapter activity from query execution to network traffic, use Logfile and Verbosity. The examples of common connection errors below show how to use these properties to get more context. Contact the support team for help tracing the source of an error or circumventing a performance issue.

• Authentication errors: Typically, recording a Logfile at Verbosity 4 is necessary to get full details on an authentication error.

• Queries time out: A server that takes too long to respond will exceed the adapter's client-side timeout. Often, setting the Timeout property to a higher value will avoid a connection error. Another option is to disable the timeout by setting the property to 0. Setting Verbosity to 2 will show where the time is being spent.


SQL Compliance |25

• The certificate presented by the server cannot be validated: This error indicates that the adapter cannot validate the server's certificate through the chain of trust. If you are using a self-signed certificate, there is only one certificate in the chain.

To resolve this error, you must verify yourself that the certificate can be trusted and specify to the adapter that you trust the certificate. One way you can specify that you trust a certificate is to add the certificate to the trusted system store; another is to set SSLServerCert.

SQL Compliance

The Splunk Adapter supports several operations on data, including querying, deleting, modifying, and inserting.

SELECT Statements

See SELECT Statements for a syntax reference and examples.

See Data Model for information on the capabilities of the Splunk API.

INSERT Statements

See INSERT Statements for a syntax reference and examples.

UPDATE Statements

The primary key Id is required to update a record. See UPDATE Statements for a syntax reference and examples.

DELETE Statements

The primary key Id is required to delete a record. See DELETE Statements for a syntax reference and examples.

EXECUTE Statements

Use EXECUTE or EXEC statements to execute stored procedures. See EXECUTE Statements for a syntax reference and examples.

Names and Quoting

• Table and column names are considered identifier names; as such, they are restricted to the following characters: [A-Z, a-z, 0-9, _:@].


26 | SQL Compliance

• To use a table or column name with characters not listed above, the name must be quoted using double quotes ("name") in any SQL statement.

• Strings must be quoted using single quotes (e.g., 'John Doe').

Transactions and Batching

Transactions are not currently supported.

Additionally, the adapter does not support batching of SQL statements. To execute multiple commands, you can create multiple instances and execute each separately.

SELECT Statements

A SELECT statement can consist of the following basic clauses.

• SELECT

• INTO

• FROM

• JOIN

• WHERE

• GROUP BY

• HAVING

• UNION

• ORDER BY

• LIMIT

SELECT Syntax

The following syntax diagram outlines the syntax supported by the Splunk adapter: SELECT { [ TOP <numeric_literal> ] { * | { <expression> [ [ AS ] <column_reference> ] | { <table_name> | <correlation_name> } .* } [ , ... ] } [ INTO csv:// [ filename= ] <file_path> [ ;delimiter=tab ] ]


SQL Compliance |27

{ FROM <table_reference> [ [ AS ] <identifier> ] } [ WHERE <search_condition> ] [ GROUP BY <column_reference> [ , ... ] [ HAVING <search_condition> ] [ ORDER BY <column_reference> [ ASC | DESC ] [ NULLS FIRST | NULLS LAST ] ] [ LIMIT <expression> ] } | SCOPE_IDENTITY()

<expression> ::= | <column_reference> | @ <parameter> | ? | COUNT( * | { [ DISTINCT ] <expression> } ) | { AVG | MAX | MIN | SUM | COUNT } ( <expression> ) | NULLIF ( <expression> , <expression> ) | COALESCE ( <expression> , ... ) | CASE <expression> WHEN { <expression> | <search_condition> } THEN { <expression> | NULL } [ ... ] [ ELSE { <expression> | NULL } ] END | <literal> | <sql_function>

<search_condition> ::= { <expression> { = | < | > | >= | <= | != | AND | OR | NOT | IN | IS | NULL | IS | NOT | NULL } [ <expression> ] } [ { AND | OR } ... ]

Examples

1. Return all columns:

SELECT * FROM DataModels

2. Rename a column:

SELECT "Owner" AS MY_Owner FROM DataModels

3. Cast a column's data as a different data type:

SELECT CAST(DatasetLimiting AS VARCHAR) AS Str_DatasetLimiting FROM DataModels


28 | SQL Compliance

4. Search data:

SELECT * FROM DataModels WHERE Id = 'SampleDataset';

5. The Splunk APIs support the following operators in the WHERE clause: =, <, >, >=, <=, !=, AND, OR, NOT, IN, IS, NULL, IS, NOT, NULL.

SELECT * FROM DataModels WHERE Id = 'SampleDataset';

6. Return the number of items matching the query criteria:

SELECT COUNT(*) AS MyCount FROM DataModels

7. Summarize data:

SELECT Owner, MAX(DatasetLimiting) FROM DataModels GROUP BY Owner

See Aggregate Functions for details.

1. Sort a result set in ascending order:

SELECT Name, Owner FROM DataModels ORDER BY Owner ASC

SELECT INTO Statements

You can use the SELECT INTO statement to export formatted data to a file.

Data Export with an SQL Query

The following query exports data into a file formatted in comma-separated values (CSV): boolean ret = stat.execute("SELECT Name, Owner INTO "csv://c:/DataModels.txt" FROM "DataModels" WHERE Id = 'SampleDataset'");System.out.println(stat.getUpdateCount()+" rows affected");

You can specify other file formats in the URI. The following example exports tab-separated values: Statement stat = conn.createStatement();boolean ret = stat.execute("SELECT * INTO "DataModels" IN 'csv://filename=c:/DataModels.csv;delimiter=tab' FROM "DataModels" WHERE Id = 'SampleDataset'");System.out.println(stat.getUpdateCount()+" rows affected");

INSERT Statements

To create new records, use INSERT statements.


SQL Compliance |29

INSERT Syntax

The INSERT statement specifies the columns to be inserted and the new column values. You can specify the column values in a comma-separated list in the VALUES clause, as shown in the following example:

INSERT INTO <table_name> ( <column_reference> [ , ... ] )VALUES ( { <expression> | NULL } [ , ... ] )

<expression> ::= | @ <parameter> | ? | <literal>

You can use the executeUpdate method of the Statement and PreparedStatement classes to execute data manipulation commands and retrieve the rows affected.

String cmd = "INSERT INTO DataModels (Owner) VALUES (?)";PreparedStatement pstmt = connection.prepareStatement(cmd);pstmt.setString(1, "user");int count = pstmt.executeUpdate();System.out.println(count+" rows were affected");connection.close();

UPDATE Statements

To modify existing records, use UPDATE statements.

Update Syntax

The UPDATE statement takes as input a comma-separated list of columns and new column values as name-value pairs in the SET clause, as shown in the following example:

UPDATE <table_name> SET { <column_reference> = <expression> } [ , ... ] WHERE { Id = <expression> } [ { AND | OR } ... ]



30 | SQL Compliance

You can use the executeUpdate method of the Statement or PreparedStatement classes to execute data manipulation commands and retrieve the rows affected, as shown in the following example:

String cmd = "UPDATE DataModels SET Owner='user' WHERE Id = ?";PreparedStatement pstmt = connection.prepareStatement(cmd);pstmt.setString(1, "1");int count = pstmt.executeUpdate();System.out.println(count + " rows were affected");connection.close();

DELETE Statements

To delete information from a table, use DELETE statements.

DELETE Syntax

The DELETE statement requires the table name in the FROM clause and the row's primary key in the WHERE clause, as shown in the following example:

<delete_statement> ::= DELETE FROM <table_name> WHERE { Id = <expression> } [ { AND | OR } ... ]


You can use the executeUpdate method of the Statement or PreparedStatement classes to execute data manipulation commands and retrieve the number of affected rows, as shown in the following example:

Connection connection = DriverManager.getConnection("jdbc:splunk:user=MyUserName;password=MyPassword;URL=MyURL;",);String cmd = "DELETE FROM DataModels WHERE Id = ?";PreparedStatement pstmt = connection.prepareStatement(cmd);pstmt.setString(1, "1");int count=pstmt.executeUpdate();connection.close();

EXECUTE Statements

To execute stored procedures, you can use EXECUTE or EXEC statements.


Data Model |31

EXEC and EXECUTE assign stored procedure inputs, referenced by name, to values or parameter names.

Stored Procedure Syntax

To execute a stored procedure as an SQL statement, use the following syntax:

{ EXECUTE | EXEC } <stored_proc_name> { [ @ ] <input_name> = <expression>} [ , ... ]


Example Statements

Reference stored procedure inputs by name:

EXECUTE my_proc @second = 2, @first = 1, @third = 3;

Execute a parameterized stored procedure statement:

EXECUTE my_proc second = @p1, first = @p2, third = @p3;

Data Model

The adapter models Splunk reports, searches, datasets, and data models as tables in a relational database that you can read from/write to with SQL-92 queries.

Dynamic Schema Generation

You can work with all of the tables in your account: when you connect the adapter retrieves the metadata from Splunk and dynamically reflects any changes in the table schemas.

You can call the CreateSchema stored procedure to persist a static schema across connections. The stored procedure saves the schema to a text file; the text file has a simple format that also makes schemas easy to customize.


32 | Data Model

Tables

See Tables for more details on updating and querying datasets, data models, and searches.

Views

The adapter also surfaces data through Views representing the following Splunk objects:

• Reports: See LookUpReport

for an example of a view representing a saved report in Splunk.

• Data-model datasets: See AlertsInInternalServer

for an example of a view representing a dataset. See Datasetss to retrieve a list of dataset views.

• Table-type datasets: See UploadedModel

for an example of a view representing a table dataset in Splunk.

Tables

The adapter exposes tables for data sources that support both retrieving and updating data.

Generally, querying Splunk tables is the same as querying a table in a relational database. The following sections provide Splunk-specific information on querying the tables. For example, any columns required in the WHERE clause and fields required to insert.

Splunk Adapter Tables

DataModels

Create, query, update, and delete data models in Splunk.

Name Description

DataModels Create, query, update, and delete data models in Splunk.

Datasets Create, query, update, and delete datasets in Splunk.

SearchJobs Create, query, update, and delete search jobs in Splunk.


Data Model |33

Select

The adapter will use the Splunk API to process search criteria that refer to the Id column. This column supports server-side processing for the = operator. The adapter processes other filters client-side within the adapter.

For example, the following query is processed server side by the Splunk APIs:

SELECT * FROM DataModels WHERE Id = 'SampleModel'

Insert

The Id column is the minimum requirement for an insert. In an insert, the DataModels table allows only the Id and Acceleration columns.

INSERT INTO DataModels (Id, Acceleration) VALUES ('initialname', '{"enabled":false,"earliest_time":"","hunk.file_format":"","hunk.dfs_block_size":0,"hunk.compression_codec":""}' )

Update

The DataModels table allows updates for the Acceleration column when Id is specified. You can also set the Provisional pseudocolumn.

UPDATE DataModels SET Provisional = 'true', Acceleration = '{"enabled":false,"earliest_time": "-1mon", "cron_schedule": "0 */12 * * *","hunk.file_format":"","hunk.dfs_block_size":0,"hunk.compression_codec":""}' WHERE Id = 'initialname'

Delete

The DataModels table allows deleting a record when Id is specified.

DELETE FROM Datamodels WHERE Id = 'initialname'

Columns

Name Type ReadOnly References Description

Id [KEY] String False Id of the data model.

LinkId String True Link of the data model.


34 | Data Model

Disabled Boolean True Indicates if the data model is disabled/enabled.

UpdatedAt Datetime True Datetime of the last update of the data model.

Description String True Description of the data model.

Name String True The name displayed for the data model in Splunk.

Author String True Splunk user who created the data model.

App String True Splunk app where the data model is shared.

Owner String True Splunk user who owns the data model.

CanShareApp

Boolean True Boolean indicating whether the data model can be shared in an app.

CanShareGlobal

Boolean True Boolean indicating whether the data model can be shared globally.

CanShareUser

Boolean True Boolean indicating whether the data model can be shared by the user.

CanWrite Boolean True Boolean indicating whether the data model can be extended by the user.

Modifiable Boolean True Boolean indicating whether the data model can be modified.

Removable Boolean True Boolean indicating whether the data model can be removed.

Acceleration String False Acceleration settings for the data model. Supply JSON to specify any or all of the following settings: enabled (true or false), earliest_time (time modifier), or cron_schedule (cron string).


Data Model |35

AccelerationAllowed

Boolean True Boolean indicating that acceleration is allowed or not for the data model.

AccelerationHunkCompression

String True Specifies the compression codec to be used for the accelerated orc or parquet format files.

DatasetCommands

String True Data model commands.

DatasetDescription

String True The JSON describing the data model.

DatasetCurrentCommand

Integer True Current command of the data model.

DatasetEarliestTime

Datetime True Earliest time of data model events being processed.

DatasetLatestTime

Datetime True Latest time of data model events being processed.

DatasetDiversity

String True Diversity of events being processed.

DatasetLimiting

Integer True Limitations of events being processed.

DatasetMode String True Search mode events being processed.

DatasetSampleRatio

String True Sample ratio of the data model.

DatasetFields String True Indexed fields the data model has.

DatasetType String True Dataset type.

Type String True Data model type.

Digest String True Content digest type.

TagsWhitelist

String True Whitelist of data model tags.

ReadPermitions

String True Permissions to read this data model.


36 | Data Model

Pseudo-Columns

Pseudo column fields are used in the WHERE clause of SELECT statements and offer a more granular control over the tuples that are returned from the data source.

Datasets

Create, query, update, and delete datasets in Splunk.

Select

The Datasets table requires DataModelId in the WHERE clause. The DataModelId column supports server-side processing for the = operator. The adapter processes other search criteria client-side within the adapter.

SELECT * FROM DataSets WHERE DataModelId = 'SampleModel'

Insert

Splunk allows inserts only when DataModelId, ParentName, and ObjectName are all specified.

INSERT INTO [Datasets] (ObjectName, ParentName, DataModelId) VALUES ('SampleSet', 'BaseEvent','SampleModel')

WritePermitions

String True Permissions to write to this data model.

Sharing String True Data model sharing type.

Username String True Username of the Splunk user.

Name Type Description

Provisional Boolean Indicates whether the data model is provisional. Provisional data models are not saved. Specify true to validate a data model before saving it.

Rows@Next String This is used to page through multiple pages of results and should not be set manually.


Data Model |37

Update

Whe Datasets table allows updates when DataModelId is specified. The columns that can be updated in this case are the following: Description and DisplayName.

When ObjectName is also specified, you can update the following columns: ObjectDisplayName, ParentName, Comment, Fields, Calculations, Constraints, Lineage, ObjectSearchNoFields, ObjectSearch, AutoextractSearch, PreviewSearch, AccelerationSearch, BaseSearch, and TsidxNamespace.

UPDATE Datasets SET Description = 'model description' , DisplayName = 'Model Display Name' WHERE DataModelId = 'SampleModel'

UPDATE Datasets SET ParentName = 'BaseEvent', BaseSearch = '| search (index=* OR index=_*) | fields _time, RootObject', AccelerationSearch = ' search (index=* OR index=_*) ' WHERE DataModelId = 'SampleModel' AND ObjectName = 'SampleSet'

Delete

Datasets can be deleted by providing the DataModelId and the ObjectName of the dataset.

DELETE FROM Datasets WHERE DataModelId = 'SampleModel' AND ObjectName = 'SampleSet'

Columns


ObjectName [KEY]

String False Name of the dataset object.

DatamodelId [KEY]

String False DataModels.Id

Id of the data model the object belongs to.

DisplayName String False Name of the data model the object belongs to.

Description String False Dataset description.

ObjectNameList String True List of the objects in the data model.

ObjectDisplayName

String False Name displayed in Splunk for the object.


38 | Data Model

Pseudo-Columns


ParentName String False Name of the Parent Event.

Comment String False Dataset comments.

Fields String False Dataset events indexed fields.

Calculations String False Saved calculations for dataset fields.

Constraints String False Saved constraints for dataset fields.

Lineage String False Dataset lineage.

ObjectSearchNoFields

String False Object search query without fields.

ObjectSearch String False Saved search query for the object.

AutoextractSearch

String False Search query for autoextraction.

PreviewSearch String False Search preview query.

AccelerationSearch

String False Search query including acceleration.

BaseSearch String False Basic search query.

TsidxNamespace

String False Allocated namespace.

EventBased Integer True Number of Event-Based objects in the data model.

TransactionBased

Integer True Number of Transaction-Based objects in the data model.

SearchBased Integer True Number of Search-Based objects in the data model.


Data Model |39

SearchJobs

Create, query, update, and delete search jobs in Splunk.

Select

The adapter will use the Splunk APIs to process the search Id (Sid) criteria specified in the WHERE clause. The Sid column supports server-side processing for the = operator. The adapter processes other search criteria client-side within the adapter. SELECT * FROM SearchJobsSELECT * FROM SearchJobs WHERE Sid = '123456789.1234'

Insert

Splunk allows inserts only when EventSearch is specified. You can insert the Custom, EarliestTime, LatestTime, Label, and StatusBuckets columns and all pseudocolumns. INSERT Into SearchJobs (Custom, EventSearch, LatestTime, Timeout) VALUES ('custom1=test1, custom2=test2', ' from datamodel SampleModel', 'now', '60')

Update

The SearchJobs table allows updates of the Custom column only when Sid is specified. UPDATE SearchJobs SET Custom = 'custom1=test3, custom2=test4' WHERE sid = '123456789.1234'

Delete

SearchJobs can be deleted by providing the Sid. DELETE FROM SearchJobs WHERE Sid = '123456789.1234'

Columns





40 | Data Model

Sid [KEY] String False The search Id number.

EventSearch String False Subset of the entire search that is before any transforming commands.

Custom String False Custom job property. In an INSERT operation, pass the values as a comma-separated list of pairs of keys and values.

EarliestTime String False The earliest time a search job is configured to start.

LatestTime String False The latest time a search job is configured to start.

CursorTime String True The earliest time from which no events are later scanned. Can be used to indicate progress.

Delegate String True For saved searches, specifies jobs that were started by the user. Defaults to scheduler.

DiskUsage Long True The total amount of disk space used, in bytes.

DispatchState String True The state of the search. Can be any of QUEUED, PARSING, RUNNING, PAUSED, FINALIZING, FAILED, or DONE.

DoneProgress Double True A number between 0 and 1.0 that indicates the approximate progress of the search. doneProgress = (latestTime-cursorTime) / (latestTime-earliestTime)

DropCount Integer True For real-time searches only, the number of possible events that were dropped due to the rt_queue_size (defaults to 100000).

EventAvailableCount

Integer True The number of events that are available for export.


Data Model |41

EventCount Integer True The number of events returned by the search.

EventFieldCount

Integer True The number of fields found in the search results.

EventIsStreaming

Boolean True Indicates if the events of this search are being streamed.

EventIsTruncated

Boolean True Indicates if the events of the search are not stored, making them unavailable from the events endpoint for the search.

EventPreviewableCount

Integer True Number of in-memory events that are not yet committed to disk.

EventSorting String True Indicates if the events of this search are sorted, and in which order.

IsDone Boolean True Indicates if the search has completed.

IsEventsPreviewEnabled

String True Indicates if the timeline_events_preview setting is enabled in limits.conf.

IsFailed Boolean True Indicates if there was a fatal error executing the search. For example, invalid search string syntax.

IsFinalized Boolean True Indicates if the search was finalized (stopped before completion).

IsPaused Boolean True Indicates if the search is paused.

IsPreviewEnabled

Boolean True Indicates if previews are enabled.

IsRealTimeSearch

Boolean True Indicates if the search is a real-time search.

IsRemoteTimeline

Boolean True Indicates if the remote timeline feature is enabled.


42 | Data Model

IsSaved Boolean True Indicates that the search job is saved on disk. Search artifacts are saved on disk for 7 days from the last time that the job was viewed or touched.

IsSavedSearch Boolean True Indicates if this is a saved search run using the scheduler.

IsZombie Boolean True Indicates if the process running the search died without finishing the search.

Keywords String True All positive keywords used by this search. A positive keyword is a keyword that is not in a NOT clause.

Label String False Custom name created for this search.

Messages String True Errors and debug messages.

NumPreviews Integer True Number of previews generated so far for this search job.

Performance String True A representation of the execution costs.

Priority Integer True An integer between 0-10 that indicates the search priority.

RemoteSearch String True The search string that is sent to every search peer.

ReportSearch String True If reporting commands are used, the reporting search.

ResultCount Integer True The total number of results returned by the search. In other words, this is the subset of scanned events (represented by the ScanCount) that actually matches the search terms.

ResultIsStreaming

Boolean True Indicates if the final results of the search are available using streaming (for example, no transforming operations).


Data Model |43

Pseudo-Columns


ResultPreviewCount

Integer True The number of result rows in the latest preview results.

RunDuration Decimal True Time in seconds that the search took to complete.

ScanCount Integer True The number of events that are scanned or read off disk.

SearchEarliestTime

Datetime True Specifies the earliest time for a search, as specified in the search command rather than the EarliestTime parameter. It does not snap to the indexed data time bounds for all-time searches.

SearchLatestTime

Datetime True Specifies the latest time for a search, as specified in the search command rather than the LatestTime parameter. It does not snap to the indexed data time bounds for all-time searches.

SearchProviders

String True A list of all the search peers that were contacted.

StatusBuckets Integer False Maximum number of timeline buckets.

TTL String True The time to live, or the time before the search job expires after it completes.



44 | Data Model

SearchMode String Searching mode, realtime or normal. If set to realtime, the search runs over the live data.

The allowed values are normal, realtime.

EnableLookups

Boolean Indicates whether lookups should be applied to events.

AutoPause Integer If specified, the search job pauses after this many seconds of inactivity. (0 means never autopause.)

AutoCancel Integer If specified, the job automatically cancels after this many seconds of inactivity. (0 means never autocancel.)

AdhocSearchLevel

Integer Specify a search mode. Use one of the following search modes: verbose, fast, or smart.

The allowed values are verbose, fast, smart.

ForceBundleReplication

Boolean Specifies whether this search should cause (and wait depending on the value of SyncBundleReplication) for bundle synchronization with all search peers.

IndexEarliest String Specify a time string. Sets the earliest inclusive time bounds for the search, based on the index time bounds.

IndexLatest String Specify a time string. Sets the latest exclusive time bounds for the search, based on the index time bounds.

IndexedRealtime

Boolean Indicates whether or not to use the indexed-realtime mode for real-time searches.

IndexedRealtimeOffset

Integer Sets disk sync delay for indexed real-time search (seconds).

MaxCount Integer The number of events that can be accessible in any given status bucket.

MaxTime Integer Comma-separated list of (possibly wildcarded) servers from which raw events should be pulled.

Namespace String The application namespace in which to restrict searches.

Now String Specify a time string to set the absolute time used for any relative time specifier in the search. Defaults to the current system time. You can specify a relative time modifier for this parameter. For example, specify +2d to specify the current time plus two days.


Data Model |45

ReduceFrequency

Integer Determines how frequently to run the MapReduce reduce phase on accumulated map values.

ReloadMacros Boolean Specifies whether to reload macro definitions from the configuration file.

RemoteServerList

Integer The number of seconds to run this search before finalizing. Specify 0 to never finalize.

ReplaySpeed Integer Indicate a real-time search replay speed factor. For example, 1 indicates normal speed, 0.5 indicates half of normal speed, and 2 indicates twice as fast as normal.

ReplayStartTime

String Relative wall-clock start time for the replay.

ReplayEndTime

String Relative end time for the replay clock. The replay stops when the clock time reaches this time.

ReuseMaxSecondsAgo

Integer Specifies the number of seconds ago to check when an identical search is started and return the search Id of the job instead of starting a new job.

RequiredField String Adds a required field to the search.

RealTimeBlocking

Boolean For a real-time search, indicates if the indexer blocks if the queue for this search is full.

RealTimeIndexFilter

Boolean For a real-time search, indicates if the indexer prefilters events.

RealTimeMaxBlockSecs

Integer For a real-time search with RealTimeBlocking set to true, the maximum time to block. Specify 0 to indicate no limit.

RealTimeQueueSize

Integer For a real-time search, the queue size (in events) that the indexer should use for this search.

Timeout Integer The number of seconds to keep this search after processing has stopped.

SyncBundleReplication

String Specifies whether this search should wait for bundle replication to complete.



46 | Data Model

Views

Views are composed of columns and pseudo columns. Views are similar to tables in the way that data is represented; however, views do not support updates. Data sources that are represented as views are typically read-only data sources. Often, a stored procedure is available to update the data if such functionality is applicable to the data source.

Queries can be executed against a view as if it were a normal table, and the data that comes back is similar in that regard. To find out more about tables and stored procedures, please navigate to their corresponding entries in this help document.

Splunk Adapter Views

AlertsInInternalServer

A dataset object in the example InternalServer data model.

Select

This is an example of a dataset view. These views are generated from dataset objects inside a data model. The adapter will use the Splunk APIs to process the following query components; the adapter processes other parts of the query client-side in memory.

All columns support server-side processing for the following operators and functions:

• Operators: =, <, >, >=, <=, IN, IS NULL, IS NOT NULL, NOT

• Functions: AVG, SUM, MIN, MAX, COUNT, STDEV, STDEVP, VAR, VARP

LIMIT, ORDER BY, GROUP BY, and HAVING are also processed server-side. An exception is the case when in the selected columns, there are fields that are not in the GROUP BY, and GROUP BY, criteria, and limiting are handled client-side.

Name Description

AlertsInInternalServer A dataset object in the example InternalServer data model.

LookUpReport An example lookup report representing a view based on a saved report in Splunk.

UploadedModel An example of a table object inside a data model.


Data Model |47

In the case when an unsupported criteria or function is used, all processing will be completed client-side (except selecting specified fields). This is also the case when a SELECT statement has a column that is not in the GroupBy clause.

For example, the adapter uses the Splunk APIs to process the following queries.

SELECT Component, Timeendpos as Timeend FROM [AlertsInInternalServer] WHERE Component = 'Saved' OR EventType != '' AND Priority IS NOT NULL AND Linecount NOT IN ('1', '2') ORDER BY Priority DESC LIMIT 5

SELECT AVG(Suppressed), Priority FROM [AlertsInInternalServer] GROUP BY Priority HAVING AVG(Suppressed) > 0


48 | Data Model

Columns


_time Datetime

component String

date_hour Int

date_mday Int

date_minute Int

date_month String

date_second Int

date_wday String

date_year Int

date_zone Int

digest_mode Int

dispatch_time Int

host String

linecount Int

log_level String

priority String

punct String

savedsearch_id String

scheduled_time Int

search_type String

server_alert_actions String

server_app String


Data Model |49

Pseudo-Columns


server_message String

server_result_count Int

server_run_time Double

server_savedsearch_name

String

server_sid String

server_status String

server_user String

source String

sourcetype String

splunk_server String

suppressed Int

thread_id String

timeendpos Int

timestartpos Int

window_time Int


Rows@Next String Identifier for the next page of results. Do not set this value manually.


50 | Data Model

LookUpReport

An example lookup report representing a view based on a saved report in Splunk.

Select

This is an example of a report view. These views are generated from saved reports in Splunk.

The adapter will use the Splunk APIs to process the following query components; the adapter processes other parts of the query client-side in memory.

Saved Search

Runs a saved search, or report, and returns the search results of a saved search. If the search contains replacement placeholder terms, such as $replace_me$, the search processor replaces the placeholders with the strings you specify.

For example:

• SELECT * FROM mysearch WHERE replace_me='value'

Will generate the following search statement:

• | savedsearch mysearch replace_me="value"

All replacement placeholder terms will be dynamic and saved as Pseudo-Columns.

All columns support server-side processing for the following operators and functions:

• Operators: =, <, >, >=, <=, IN, IS NULL, IS NOT NULL, NOT, LIKE



In the case when an unsupported criteria or function is used, all processing will be completed client-side (except selecting specified fields). This is also the case when a SELECT statement has a column that is not in the GROUP BY clause.

For example, the adapter processes the following queries server-side: SELECT Country, Subregion as Sub FROM LookUpReport WHERE Iso2 != '123' OR continent = 'Europe' AND iso3 NOT IN ('example_1', 'example_2') ORDER BY Country DESC LIMIT 5

SELECT AVG(Iso2), Subregion FROM LookUpReport GROUP BY Subregion HAVING AVG(Iso2) > 0


Data Model |51

Columns

Pseudo-Columns


UploadedModel

An example of a table object inside a data model.

Select

This is an example of a view generated from a table object inside a data model. The adapter will use the Splunk APIs to process the following query components; the adapter processes other parts of the query client-side in memory.

All columns support server-side processing for the following operators and functions.

• Operators: =, <, >, >=, <=, IN, IS NULL, IS NOT NULL, NOT


continent String

country String

iso2 String

iso3 String

region_un String

region_wb String

subregion String




52 | Data Model



In the case when an unsupported criteria or function is used, all processing will be completed client-side (except selecting specified fields). This is also the case when a SELECT statement has a column that is not in the GROUP BY clause.

For example, the following queries are processed server side:

SELECT Component, Timeendpos as Timeend FROM [UploadedModel] WHERE Component = 'Saved' OR DEST_CITY_MARKET_ID != '' AND DEST_AIRPORT_ID NOT IN ('1', '2') ORDER BY ORIGIN_AIRPORT_ID DESC LIMIT 5

SELECT AVG(DEST_AIRPORT_ID), ORIGIN_AIRPORT_ID FROM [UploadedModel] GROUP BY ORIGIN_AIRPORT_ID HAVING AVG(DEST_AIRPORT_ID) > 0


Data Model |53

Columns

Pseudo-Columns



_time Datetime

DEST_AIRPORT_ID Int

DEST_AIRPORT_SEQ_ID Int

DEST_CITY_MARKET_ID Int

host String

linecount Int

ORIGIN_AIRPORT_ID Int

ORIGIN_AIRPORT_SEQ_ID Int

ORIGIN_CITY_MARKET_ID Int

punct String

source String

sourcetype String

splunk_server String

timestamp String




54 | Data Model

Stored Procedures

Stored procedures are available to complement the data available from the Data Model. It may be necessary to update data available from a view using a stored procedure because the data does not provide for direct, table-like, two-way updates. In these situations, the retrieval of the data is done using the appropriate view or table, while the update is done by calling a stored procedure. Stored procedures take a list of parameters and return back a dataset that contains the collection of tuples that constitute the response.

Splunk Adapter Stored Procedures

CreateSchema

Creates a schema file for the specified table or view.

Input

Result Set Columns

Name Description

CreateSchema Creates a schema file for the specified table or view.

Name Type Required Description

TableName String True The name of the table or view.

FileName String True The full file path and name of the schema to generate. The Location connection property specifies the default output directory.


Result String Returns Success or Failure.


TIBCO Data Virtualization Splunk Adapter Guide

Documents

Transcript of TIBCO Data Virtualization Splunk Adapter Guide