Study of algorithms that

• improve their performance P
• at some task T

training data set, validation data set, test data set.

• with experience E

• supervised learning

Fitting some data to a function or function approximation.

• unsupervised learning

Figuring out what the data is without any feedback. For instance, if we were given many data points, we could group them by similarity, or perhaps determine which variables are better than others.

\[H = {H|h: X \to Y}\]

• Linear Regression
• Logistic Regression
• Neural Networks
• Support Vector Machines
• K-means Clustering
• Principal Components Analysis
• Anomaly Detection
• Collaborative Filtering
• Object Recognition

To establish notation for future use, we’ll use x(i) to denote the “input” variables (living area in this example), also called input features, and y(i) to denote the “output” or target variable that we are trying to predict (price). (x(i),y(i)) is called a training example. m—is called a training set.

\(J(\Theta_1,\Theta_2)\) contour is the bow projected on the 2D surface. A contour plot is a graph that contains many contour lines. A contour line of a two variable function has a constant value at all points of the same line.

• Loss function is usually a function defined on a data point, prediction and label, and measures the penalty. For example:
  • square loss l(f(xi|θ),yi)=(f(xi|θ)−yi)2l(f(xi|θ),yi)=(f(xi|θ)−yi)2, used in linear regression
  • hinge loss l(f(xi|θ),yi)=max(0,1−f(xi|θ)yi)l(f(xi|θ),yi)=max(0,1−f(xi|θ)yi), used in SVM
  • 0/1 loss l(f(xi|θ),yi)=1⟺f(xi|θ)≠yil(f(xi|θ),yi)=1⟺f(xi|θ)≠yi, used in theoretical analysis and definition of accuracy
• Cost function is usually more general. It might be a sum of loss functions over your training set plus some model complexity penalty (regularization). For example:
  • Mean Squared Error MSE(θ)=1N∑Ni=1(f(xi|θ)−yi)2MSE(θ)=1N∑i=1N(f(xi|θ)−yi)2
  • SVM cost function SVM(θ)=∥θ∥2+C∑Ni=1ξiSVM(θ)=‖θ‖2+C∑i=1Nξi (there are additional constraints connecting ξiξi with CC and with training set)
• Objective function is the most general term for any function that you optimize during training. For example, a probability of generating training set in maximum likelihood approach is a well defined objective function, but it is not a loss function nor cost function (however you could define an equivalent cost function). For example:
  • MLE is a type of objective function (which you maximize)
  • Divergence between classes can be an objective function but it is barely a cost function, unless you define something artificial, like 1-Divergence, and name it a cost

• <2017-08-14 Mon>

linear regression with one variable is also called simple regression. The X variable is called the predictor, Y is called the dependent.

• some statistics:
  • t-statistics off value according to the \(/epsilon\).
  • p value the probability of the hypothesis that \(/beta_1\) is 0.
  • \(R^2\) the confidence level that \(/beta_1\) is approximately estimated.
  • RSS residual sum of squares
  • RSE residual standard error.

• Vector
• Matrix

Support vector machines (SVMs) are a set of related supervised learning methods used for classification and regression. Given a set of training examples, each marked as belonging to one of two categories, an SVM training algorithm builds a model that predicts whether a new example falls into one category or the other.

如何利用熵分类： 怎么样贪心地分类让熵以最快的速度降低。

信息增益： 给定特征A的信息而使得类X的信息的不确定性减少的程度。

• first principal component
• second principal component

Cluster analysis is the assignment of a set of observations into subsets (called clusters) so that observations within the same cluster are similar according to some predesignated criterion or criteria, while observations drawn from different clusters are dissimilar. Different clustering techniques make different assumptions on the structure of the data, often defined by some similarity metric and evaluated for example by internal compactness (similarity between members of the same cluster) and separation between different clusters. Other methods are based on estimated density and graph connectivity. Clustering is a method of unsupervised learning, and a common technique for statistical data analysis.

1. Choose a class of model by importing the appropriate estimator class from ScikitLearn.

In[6]: from sklearn.linear_model import LinearRegression

1. Choose model hyperparameters by instantiating this class with desired values.

In[7]: model = LinearRegression(fit_intercept=True)
model
Out[7]: LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1,
normalize=False)

1. Arrange data into a features matrix and target vector following the discussion

from before.

In[8]: X = x[:, np.newaxis]
X.shape
Out[8]: (50, 1)

1. Fit the model to your data by calling the fit() method of the model instance.

In[9]: model.fit(X, y)
Out[9]:
LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1,
normalize=False)
In[10]: model.coef_
Out[10]: array([ 1.9776566])
In[11]: model.intercept_
Out[11]: -0.90331072553111635

1. Apply the model to new data:

In[12]: xfit = np.linspace(-1, 11)
In[13]: Xfit = xfit[:, np.newaxis]
yfit = model.predict(Xfit)

• For supervised learning, often we predict labels for unknown data using the predict() method. • For unsupervised learning, we often transform or infer properties of the data using the transform() or predict() method.

1. Define the objective in business terms.

Portfolio optimization is the process of choosing the proportions of various assets to be held in a portfolio, in such a way as to make the portfolio better than any other according to some criterion. The criterion will combine, directly or indirectly, considerations of the expected value of the portfolio's rate of return as well as of the return's dispersion and possibly other measures of financial risk.

1. How will your solution be used?

Often, portfolio optimization takes place in two stages: optimizing weights of asset classes to hold, and optimizing weights of assets within the same asset class. An example of the former would be choosing the proportions placed in equities versus bonds, while an example of the latter would be choosing the proportions of the stock sub-portfolio placed in stocks X, Y, and Z. Equities and bonds have fundamentally different financial characteristics and have different systematic risk and hence can be viewed as separate asset classes; holding some of the portfolio in each class provides some diversification, and holding various specific assets within each class affords further diversification. By using such a two-step procedure one eliminates non-systematic risks both on the individual asset and the asset class level.

1. What are the current solutions/workarounds (if any)?

Markowitz Modern Portfolio Theory, Blacklitterman Model, Risk Parity.

1. How should you frame this problem (supervised/unsupervised, online/offline,etc.)?

reinforcement learning, offline.

1. How should performance be measured?

Return(percentage change) and Risk(standard deviation or the return). In addition to the traditional measure, standard deviation, or its square (variance), which are not robust risk measures, other measures include the Sortino ratio and the CVaR (Conditional Value at Risk).

1. Is the performance measure aligned with the business objective?

Often portfolio optimization is done subject to constraints.

1. What would be the minimum performance needed to reach the business objective?
2. What are comparable problems? Can you reuse experience or tools?
3. Is human expertise available?
4. How would you solve the problem manually?
5. List the assumptions you (or others) have made so far.
6. Verify assumptions if possible.

Note: automate as much as possible so you can easily get fresh data.

1. List the data you need and how much you need.
2. Find and document where you can get that data.
3. Check how much space it will take.
4. Check legal obligations, and get authorization if necessary.
5. Get access authorizations.
6. Create a workspace (with enough storage space).
7. Get the data.
8. Convert the data to a format you can easily manipulate (without changing the data itself).
9. Ensure sensitive information is deleted or protected (e.g., anonymized).
10. Check the size and type of data (time series, sample, geographical, etc.).
11. Sample a test set, put it aside, and never look at it (no data snooping!).

Note: try to get insights from a field expert for these steps.

1. Create a copy of the data for exploration (sampling it down to a manageable size

if necessary).

1. Create a Jupyter notebook to keep a record of your data exploration.
2. Study each attribute and its characteristics:

• Name • Type (categorical, int/float, bounded/unbounded, text, structured, etc.)of missing values • Noisiness and type of noise (stochastic, outliers, rounding errors, etc.) • Possibly useful for the task? • Type of distribution (Gaussian, uniform, logarithmic, etc.)

1. For supervised learning tasks, identify the target attribute(s).
2. Visualize the data.
3. Study the correlations between attributes.
4. Study how you would solve the problem manually.
5. Identify the promising transformations you may want to apply.
6. Identify extra data that would be useful (go back to “Get the Data” on page 498).
7. Document what you have learned.

Notes: • Work on copies of the data (keep the original dataset intact). • Write functions for all data transformations you apply, for five reasons: — So you can easily prepare the data the next time you get a fresh dataset — So you can apply these transformations in future projects — To clean and prepare the test set — To clean and prepare new data instances once your solution is live — To make it easy to treat your preparation choices as hyperparameters

1. Data cleaning:

• Fix or remove outliers (optional). • Fill in missing values (e.g., with zero, mean, median…) or drop their rows (or columns).

1. Feature selection (optional):

• Drop the attributes that provide no useful information for the task.

1. Feature engineering, where appropriate:

• Discretize continuous features. Decompose features (e.g., categorical, date/time, etc.). • Add promising transformations of features (e.g., log(x), sqrt(x), x², etc.). • Aggregate features into promising new features.

1. Feature scaling: standardize or normalize features.

Notes: • If the data is huge, you may want to sample smaller training sets so you can train many different models in a reasonable time (be aware that this penalizes complex models such as large neural nets or Random Forests). • Once again, try to automate these steps as much as possible.

1. Train many quick and dirty models from different categories (e.g., linear, naive Bayes, SVM, Random Forests, neural net, etc.) using standard parameters.
2. Measure and compare their performance.

• For each model, use N-fold cross-validation and compute the mean and standard deviation of the performance measure on the N folds.

1. Analyze the most significant variables for each algorithm.
2. Analyze the types of errors the models make.

• What data would a human have used to avoid these errors?

1. Have a quick round of feature selection and engineering.
2. Have one or two more quick iterations of the five previous steps.
3. Short-list the top three to five most promising models, preferring models that make different types of errors.

Notes: • You will want to use as much data as possible for this step, especially as you move toward the end of fine-tuning. • As always automate what you can. “Practical Bayesian Optimization of Machine Learning Algorithms,” J. Snoek, H. Larochelle, R. Adams (2012).

1. Fine-tune the hyperparameters using cross-validation.

• Treat your data transformation choices as hyperparameters, especially when you are not sure about them (e.g., should I replace missing values with zero or with the median value? Or just drop the rows?). • Unless there are very few hyperparameter values to explore, prefer random search over grid search. If training is very long, you may prefer a Bayesian optimization approach (e.g., using Gaussian process priors, as described by Jasper Snoek, Hugo Larochelle, and Ryan Adams).

1. Try Ensemble methods. Combining your best models will often perform better than running them individually.
2. Once you are confident about your final model, measure its performance on the test set to estimate the generalization error.

Don’t tweak your model after measuring the generalization error: you would just start overfitting the test set.

1. Document what you have done.
2. Create a nice presentation.

• Make sure you highlight the big picture first.

1. Explain why your solution achieves the business objective.
2. Don’t forget to present interesting points you noticed along the way.

• Describe what worked and what did not. • List your assumptions and your system’s limitations.

1. Ensure your key findings are communicated through beautiful visualizations or

easy-to-remember statements (e.g., “the median income is the number-one predictor of housing prices”).

1. Get your solution ready for production (plug into production data inputs, write unit tests, etc.).
2. Write monitoring code to check your system’s live performance at regular intervals and trigger alerts when it drops.

• Beware of slow degradation too: models tend to “rot” as data evolves. • Measuring performance may require a human pipeline (e.g., via a crowdsourcing service). • Also monitor your inputs’ quality (e.g., a malfunctioning sensor sending random values, or another team’s output becoming stale). This is particularly important for online learning systems.

1. Retrain your models on a regular basis on fresh data (automate as much as possible)