Automated Documentation

Just a float with an associated Symbol

Just a wrapper around a dictionary

Retrieves the symbol (the currency) of the money

Indicates if 2 Money instances have the same currency

Test if Wallets are identical. Potentially can be redefined as === and leave == if different keys are set to 0

Structures

This module provides data structures defined and used in the Backtest process.

Facilitating contract enforcement between Engines, Markets and Strategies.

Context definitions for contract between Engines, Markets and Strategies to operate with.

This context allows for arithmetic operations of portfolio and accounts, such as adding securities by doing portfolio+=security, and or removing Money from an account by doing account+=money.

A way to initialize this this context type can look like:

using AirBorne.Structures: ContextTypeB, TimeEvent
using Dates: now
start_event = TimeEvent(now(),"hi")
new_context = ContextTypeB(start_event)

c_get(context::ContextTypeA,key::Union{Symbol,String},default::Any; paramFirst::Bool=true)

Returns a value from within the extra or parameters hashmaps of the context object. Use the paramFirst attribute to determine 
whether the parameters is looked into first or not. 

This function is particularly useful when a datapoint may come from either "extra" or "parameters", "parameters" tend to contain dynamic hyperparameters
that can be modified after an optimization routing whilst "extra" contains static hyperparameters, some strategy templates will fetch hyperparameters from either.

nextDay!(context::ContextTypeA;days::Day=Day(1))

print_summary(summary::DataFrame)

Given a summary of the performance of the portfolio over time, returns plots of the performance 
summary.

# Arguments
- `summary::DataFrame`: A dataframe with the summary of the performance of the portfolio over time.
    Must be same format as the output of `summarizePerformance` function.

summarizePerformance(data::DataFrame, context::ContextTypeA;
valuationFun::Function=stockValuation,
removeWeekend::Bool=false,
keepDaysWithoutData::Bool=true,
windowSize::Int=5,
riskFreeRate::Real=0.0
)

Given an audit of the portfolio, account, events and OHLCV data it returns a summary of the performance 
of the portfolio over time.

# Arguments
- `data::DataFrame`: A dataframe with the data OHLCV_V1 data.
- `context::ContextTypeA`: Result from running the simulation in DEDS Engine.
### Optional keyword arguments
-`valuationFun::Function=stockValuation`: Stock needs to be valued to establish a notion of returns. This allows to pass custom functions for asset valuation.
-`removeWeekend::Bool=false`: Many markets close over weekend. If events are kept with lack of activity returns of 0 may be observed.
-`keepDaysWithoutData::Bool=true`: Without data the most recent  market provided will be used to estimate the value of assets. If events are kept with lack of activity returns of 0 may be observed.
-`windowSize::Int=5`: Many statistical figures are observed over sliding time windows, this allows to select the size of the timewindows by setting the number of consecutive events considered.
-`riskFreeRate::Real=0.0`: Sharpe and other metrics rely on a definition of a risk free rate. 
-`includeAccounts::Bool=true`: By default we assume that the account is not reflected in the portfolio, if accounts are included in the portfolio set *includeAccounts* to false to avoid double counting the value of money in the account.

deepPush!(list,element)

Inserts the deepcopy of an element into a collection

get_latest(df,id_symbols,sort_symbol)

Retrieves last record from a dataframe, sortying by sort_symbol and grouping by id_symbols.

```julia
get_latest(past_data,[:exchangeName,:symbol],:date)
```

get_latest_N(sdf::Union{SubDataFrame,DataFrame},by::Symbol,N::Int64; rev=false,fields::Vector=[])

This function returns a DataFrame with the first N rows of the input dataframe sorted by the column *by* amd the columns specified by *fields*. 

Using the additional parameter, *rev* the sort order gets reversed. 

Example: get the 5 largest companies in the NASDAQ screener, per sector.
```julia
	using AirBorne.ETL.NASDAQ: screener
	tickers_df = screener()
	filtered_df =tickers_df[[   x!="" ? parse(Int64, x)<2017 : false for x in tickers_df.ipoyear],["symbol","marketCap","sector"]]
	grouped_df = groupby(filtered_df,"sector")
	f(sdf)= get_latest_N(sdf,:marketCap,5;rev=true, fields = ["symbol", "marketCap"])
	result = combine(gdf,f)
```
Another example
```
a="A";b="B"

f2(sdf)= get_latest_N(sdf,:val,3)
df =DataFrame(Dict(
		"cat"=>[a,a,a,a,a,b,b,b,b,b],
		"val"=>[5,3,2,6,1,4,3,8,6,2],
		 "ix"=>[1,2,3,4,5,6,7,8,9,10],
))
combine(groupby(df,"cat"),f2)
# 6×3 DataFrame
# Row	cat	    ix	    val
#       String	Int64	Int64
# 1     A	    5	    1
# 2     A	    3	    2
# 3     A	    2	    3
# 4     B	    10	    2
# 5     B	    7	    3
# 6     B	    6	    4
```

hello_world()

Returns a string saying "Hello World!".

lagFill(df::DataFrame,col::Symbol; fill::Vector=[missing,nothing]))

Replaces all missing values in a column of a dataframe for the previous 
non-missing value.
### Arguments
-`inV::Vector`: Input Vector
-`fill::Vector`: Vector with elements to be filled. I.e., nothing, NaN, missing. By default: [missing,nothing].

Given a function (mean, variance, sharpe,...) from an 1-D array to a single element
It creates an array with same size of original with the function applied from the 
beginning of the array to the index of the output. 

!!! Tip "Performance"
	This function is not meant to be highly performant. If a function is used function is
	advised to have a specialized function to calculate its running counterpart.

# Optional Arguments
- `windowSize::Union{Int,Nothing}`: If its desired to truncate the number of past elements to be considered, this field can be set to the maximum number of past elements to take into account. This can be used for Moving Averages for example. 
- `ignoreFirst::Int`: Indicates for how many elements the operation should not be applied. In those elements "nothing" will be placed instead.

More efficient implementation of a moving average (mean running).

Recursive block diagonal

rvcat(v): Recursive vcat.

vcat has limited support for sparce matrices, this function allows to vertically concatenate 
sparse matrices.

sortStruct!(v::Vector, symbol;rev=true)

Sorts a struct by a given symbol.

sortedStructInsert!(v::Vector, x,symbol;rev=true)

Inserts a struct into a sorted Vector of structs in the right place to keep the array sorted.

Kronecker delta

This modules centralizes all features for the data pipeline of AirBorne including API
connections, cache management and data transformations as part of the data pipeline.

AssetValuation

This module provides standardized asset valuation techniques compatible with 
datastructures in this module.

covariance(assetValuesdf::DataFrame)

Calculates the covariance matrix given a Asset Value DataFrame or an Asset Return DataFrame.

logreturns(array::Vector)

Calculates the logarithmic returns in base 10 of the ratio with respect to the previous element. The return of the first element is set to 1 as the starting point.

returns(assetValuesdf::DataFrame) 

Calculates the returns of each ticker in the Asset Value DataFrame. By default as the relative percent with respect to the previous element,
the return of the first element is set to 0 as the starting point.

returns(array::Vector)

Calculates the returns as relative percent with respect to the previous element. The return of the first element is set to 0 as the starting point.

sharpe(avgReturn::Vector,variance::Vector;riskFreeRate::Real=0.0)

Calculates the sharpe ratio from 2 vectors of same length containing the mean and variance of the returns respectively.

sharpe(returns::Vector;riskFreeRate::Real=0.0,windowSize::Union{Int,Nothing}=nothing, startFrom::Int=1)

Calculates the sharpe ratio from a single vector, by calculating its mean and variance over sliding 
time windows.

stockValuation(data::DataFrame ;col::Symbol=:close,assetCol::Symbol=:assetID)

Provides the value of individual tickers (share of an equity asset) at a certain 
point in time given a OHLCV dataframe.

assetValue(data::DataFrame;method::Symbol=:last_open)

Provides a method to calculate the value of an asset given some data.

This modules centralizes caching for the AirBorne package. Containing:
- Definition of data storage procedures
- Definition of data storage formats

describe_bundles(;archive=false)

Returns the list of dictionaries describing the files for each bundle. 
It allows to retrieve information about the archived files like:
    - Status of the files (missing, corrupted, etc.)
    - Schema
    - Timestamp of storage
    - File name

gen_id()

Generates an id based on the current UTC timestamp with format "yyyy_mm_dd_H_M_S_s"

 get_cache_path()

Defines the cache path depending on the OS and environment variables.
```julia
julia> import AirBorne
julia> AirBorne.ETL.Cache.get_cache_path()
```

hello_cache()

Returns a string saying "Hello Cache!".

list_bundles()

Returns the list of bundles available in the cached folder.

In the future this function can be expanded to return information as timestamp, 
format of data in bundle among relevant metadata.

load_bundle(bundle_id::String)

Loads data from a cached bundle.

# Returns
DataFrames.DataFrame

remove_bundle(bundle_id::String; just_archive::Bool=false)

Removes bundle from cache. This is an irreversible operation. If just_archive is true it only flushes the archive folder.

store_bundle(data::DataFrames.DataFrame; bundle_id::Union{Nothing, String}=nothing, archive::Bool=true, meta::Dict=Dict(), c_meta::Dict=Dict())

Stores a dataframe in a bundle in parquet format.

**Is very important that none of the columns are of type "Any"** as the storage for this column type is not defined.

Transform is the module were standard transformation between data structure takes place.

Particular datasources should adhere to data structures defined in this module. In particular
strategies and markets may make use of data structures in this module in order to promote cross-compatibility 
between different strategies, markets and engines.

This modules provides an interface between AirBorne and Yahoo Finance API. 

For ForEx to obtain 5m definition the requested range must be within the last 60 days.

To request ForEx data replace the ticker by `codeA*codeB*"=X"` i.e. to obtain the ForEx of USD and GBP the 
ticker must be "USDGBP=X".

get_chart_data(symbol, period1, period2, freq)

This function calls the Yahoo chart API to get the OHCLV data.
The documentation for this function is based on [CryptoCoinTracker's guide](https://cryptocointracker.com/yahoo-finance/yahoo-finance-api#34a6032b7b9949a1876f4568e4961afd).

# Arguments

- `symbol::String`:  Ticker Symbol
- `period1::String`:  UNIX Timestamp indicating the start of the data requested
- `period2::String`:  UNIX Timestamp indicating the end time of the data requested 
- `freq::String`: The time interval between two data points. Can be 1m 2m 5m 15m 30m 60m 90m 1h 1d 5d 1wk 1mo 3mo.

# Returns
- r::HTTP.Messages.Response

# Examples
```julia
# This example is untestable as it requires internet connection.
julia> import AirBorne
julia> r = AirBorne.ETL.YFinance.get_chart_data("AAPL","1577836800","1580515200","1d")
```

function get_interday_data(symbols, period1, period2)

Use this function to get interday data for different tickers from Yahoo charts API.
# Arguments
- `symbols::String`:  Ticker Symbol
- `period1::String`:  UNIX Timestamp indicating the start of the data requested
- `period2::String`:  UNIX Timestamp indicating the end time of the data requested 

# Example
```julia
import AirBorne
data = AirBorne.ETL.YFinance.get_interday_data(["AAPL","GOOG"],"1577836800","1580515200")
```

hello_yfinance()

s a string saying "Hello YFinance!".

This modules provides an interface between AirBorne and NASDAQ API.

screener()

Returns the data from  [NASDAQ's screner page](https://www.nasdaq.com/market-activity/stocks/screener).

It provides a simple way also of getting a relatively large amount of US tickers.

Strategies

This module centralizes access to different market models.

A market model is tuple with a module and a struct that can be used to simulate/estimate the execution of orders.  

A market can represent a stock exchange like NYSE or secondary markets.

Assumptions made during the modelling of the market will have a direct impact on the results from backtesting,
any result from backtesting should be referenced to a strategy and a market model.

StaticMarket

The static market assumes that the prices of the assets are not affected by the strategy used. 

This is a big assumption for large orders but for smaller ones it can hold.

Order

Represents an order to the market. I.e. A stock exchange. 

TODO: Add more documentation

addJournalEntryToLedger!(ledger::Vector{Any},journal_entry::Union{DotMap,Dict})

StaticMarket method to add journal entries to ledger.

addMoneyToAccount!(account::DotMap, journal_entry)    

StaticMarket method to add money to the accounts given an entry to ledger.
The convention is that a positive ledger entry correspond to money exiting the account.

addSecurityToPortfolio!(portfolio::Union{DotMap,Dict},journal_entry::Union{DotMap,Dict})

StaticMarket method to add  securities to portfolios .

available_data(context,data)

This function determine what data is available given the context.

executeOrderCA!( context::ContextTypeA, order::Order, curdata::DataFrame; priceModel::Function=refPrice ) Default order execution method when using ContextTypeA in the simulation.

- ``:

execute_orders(
    context::ContextTypeA, data::DataFrame; executeOrder::Function=executeOrder_CA!; propagateBalanceToPortfolio::Bool=false
    )

This function updates the portfolio of the user that is stored in the variable context.

The static Market assumes that orders do not modify market attributes. Therefore orders can be executed
sequentially without consideration on how the order on one asset may affect the price on another.

-`propagateBalanceToPortfolio::Bool=false`: If the balance of the account needs to also be reflected in the portfolio 
    set this value to true and the value of *order.specs.account.currency* in the portfolio will be replaced by *order.specs.account.balance*.

expose_data(context,data)

This function determine how the data is transformed and filtered before being passed to the user.

# Arguments
-`context::ContextTypeA`: Context of the simulation.
- `data::DataFrame`: The dataframe provided to the simulation. 
### Optional keyword arguments
-`historical::Bool=true`: If true returns data up to the specified event in the context, otherwise returns just data matching the
time of the current event of the context.

genOrder(assetId::Union{String,Symbol},amount::Real; account::Any=nothing,orderType::String="MarketOrder")

Shortcut to generate market orders, in it the assetId is defined by "ExchangeID/TickerSymol", 
amount is a real number with the number of shares to be purchased, account is the account to be used to
provide the money for the transaction and order type is the type of the order.

Defines the unique identifier to an equity asset given a journal entry of the ledger

ordersForPortfolioRedistribution(
    sourcePortfolio::Dict{String, Float64}, 
    targetDistribution::Dict{String, Float64},
    assetPricing::Dict{String, Float64};
    curency_symbol::String= "FEX/USD", 
    account::Any=nothing,
    costPropFactor::Real=0,
    costPerTransactionFactor::Real=0,
    )
This function generates the orders to obtain a particular value distribution on a given portfolio and static pricing.
It can consider proportional costs by scaling the orders amount by a factor and a fixed cost for each transacted asset.
It returns the portfolio with the desired distribution and the maximum amount of value expressed in a particular currency.

-`sourcePortfolio::Dict{String, Float64}`: Dictionary with assets and how many units of them are present in a portfolio 
-`targetDistribution::Dict{String, Float64}`: Desired distribution of the total value of the portfolio across the whole shares. The values do not need to add to 1, linear scaling will be used.
-`assetPricing::Dict{String, Float64}`: Value of each share of an asset, with a corresponding value expressed in terms of a currency.
-`curency_symbol::String= "FEX/USD"`: Symbol used to represent the currency in which the transactions are goint to take place. By default dollars, it should have value 1 on the assetPricing dictionary.
-`account::Any=nothing`: Argument to be passed to the account field in the orders.
-`costPropFactor::Real=0`:  Fee rate applied to the sell or purchase of any asset proportional to the value of the transaction.
-`costPerTransactionFactor::Real=0`: Fee per transaction, every time an asset is sold/bought this fill will apply.

Given an single order and a fee structure it returns a journal entry to be passed to the ledger. 

### Arguments
- `order::Order`: The order that the fee is to be calculated for
- `feeStruct::Dict`: The fee structure containing the information for the fee calculation

### Optional Arguments
- `transactionId::Union{String,Nothing}=nothing`:Id of the original transaction
- `date::Union{DateTime,Nothing}=nothing`: Date of the transaction (or fee payment)
- `sharePrice::Union{Real,Nothing}=nothing`: Price to be paid for a single share

refPrice(cur_data::DataFrame, ticker::Union{String,Symbol}; col::Symbol=:open)

Using the current data in the market establishes the price to be paid per a unit of an asset 
(a share for example in equity).

Assuming a dataframe with one row per ticker where the ticker symbol is in the column symbol
The price is assumed to be at the column "col"

This modules centralizes all features for strategy backtesting including different simulation archetypes.

DEDS - Which stands for "Discrete Event Driven Simulation"  is a framework for backtesting
where the system moves from one event to the next one.

DEDS module hello world

run(data::DataFrame, initialize!::Function, trading_logic!::Function, execute_orders!::Function,expose_data::Function;audit=true)

Run DEDS simulation provided:
# Arguments
- `data::DataFrame`: A dataframe with the data to be provided to the function `expose_data` and `Function, execute_orders!`.
- `initialize!::Function`: initialize!(context) should receive a struct context and provide initialization for its accounts 
    and add as the next events to be processed by this function.
- `trading_logic!::Function`: `trading_logic!(context,exposed_data)` receives the context and exposed data from the market 
  and should place orders and define further events 
- `execute_orders!::Function`: execute_orders!(past_event_date, context.current_event.date, context, data) Executes orders between
    the past event and the current one, this function will modify the portfolio and the accounts from the context.
### Optional keyword arguments
- `audit::Bool=true`: If true context will its audit entry populated for each event in the simulation. 
- `max_iter::Int=10^6`: Limit the number of events processed on the simulation.
# Returns
- `verbose::Bool=false`: If true prints the date

Strategies

This module centralizes access to different trading strategy templates.

SMA (Simple Moving Average)

This is a standard strategy that can be implemented in several ways.

1. Crossover Simple Moving Average: Define 2 time windows, a long one and a short one.
If the Average during the short one is greater than over the long one this implies that the price  is going up.
So a long position is desired, however if its smaller then this indicates a falling price and a short position 
is desired.

In this Strategy an optimization of hyperparameters will be available. The optimization will need an objective function and 
maybe constraints.

The design of this strategy is inspired by the lecture 2 of Algorithmic Trading, original source obtained from  [Algorithmic Trading Society Lectures Imperial College London](https://github.com/algotradingsoc/Lectures2022/blob/main/AlgoTradeSocLectures.ipynb)

initialize!

Template for the initialization procedure, before being passed onto an engine like DEDS a preloaded
function must be defined so that the initialization function meets the engine requirements.

```julia
# Specify custom arguments to tune the behaviour of SMA
my_initialize!(context,data) = SMA.initialize!(context;...)
# Or just run with the default parameters
my_initialize!(context,data) = SMA.trading_logic!(context)
```

interday_trading_logic!(context::ContextTypeA, data::DataFrame)

Template for the trading logic algorithm, before being passed onto an engine like DEDS a preloaded
function must be defined so that the trading logic function meets the engine requirements.

```julia
# Specify custom arguments to tune the behaviour of SMA
my_trading_logic!(context,data) = SMA.trading_logic!(context,data;...)
# Or just run with the default parameters
my_trading_logic!(context,data) = SMA.trading_logic!(context,data)
```

Markowits Portfolio Optimization

It assumes the returns of the individual stocks to be a stationary stochastic process, not necessarily 
independent. Therefore it has a constant mean and variance matrix for the asset returns.


From stock prices it assumes the returns to be 
This implementation follow the one in [JuMP non-linear Portfolio Optimization](https://jump.dev/JuMP.jl/stable/tutorials/nonlinear/portfolio/)

MeanVarianceMPC

This module provides a template strategy we call Mean Variance Model Predictive Control or Mean-Variance MPC for short.

This strategy provides seeks the set of optimal portfolio distributions over time considering the cost of changing positions and 
a forecast on the expected returns and as well covariance matrix of returns.

predeterminedReturns(context::ContextTypeA,returnData::DataFrame)

Returns a sequence of tuples with (returnVector,CovarianceMatrix). Since the values of the predeterminedReturns is assumed to be known constants
the variance and convariance between assets is set to 0.

-`returnData::DataFrame`: Dataframe that has one row per time and a column per assetID at least containing the elements of *context.extra.symbolOrder*. Each 
entry on the dataframe corresponds to the return of 1 unit of the assetId of the corresponding column between the time of the previous row and its corresponding row .

To use this function context needs to have defined the following attributes:
-`context.extra.symbolOrder::Vector{String}`: The order in which the return and covariance matrix vector should be expressed.
-`context.current_event::TimeEvent`: The current event of the simulation. (Mandatory for context)
-`context.parameters.horizon::Int64`: The number of sequences to be read from returnData and transformed into tuples of return vector and variance matrices.

To use this forecast in the tradingLogicMPC! strategy
```julia
    forecastFun(context) = predeterminedReturns(context, returnsData) # returnsData must already be defined.
    custom_trading_logic!(context,data) = tradingLogicMPC!(context,data;forecastFun=forecastFun) 
```